Layer 3 - Data Management - Red

There are three kinds of data stores: file systems, databases, and directories.

File systems are hierarchical indexes designed to assist us to organize and to understand how data is arranged in a computer. Data isn't actually organized that way, but file systems are handy indexes to the actual locations which are arranged and partitioned at a low level that users are not expected to understand. By organizing and following tree-based structures, users and systems administrators can save a good deal of time and manage data in the form of text files, formatted documents, and functional applications.

Databases provide means of organizing numbers, text, and symbolic data in a variety of formats, including multimedia assets and documents. There is a constant tug-of-war between clarity and complexity in this process. Though originators of relational database models intended for clarity to result from use of their concepts, the result has been database structures featuring breathtaking complexity – with hundreds, even thousands of tables, or data arrays that incorporate complexity, but with unfathomable structural bulk. This overwhelming complexity was not foreseen by Codd, who said, “Future users of large data banks must be protected from having to know how the data is organized in the machine (the internal representation).”

Directories also play a part in the data structures of an organization. Specifically, they record profiles of users and other data, mostly associated with networks and control of resources. While database data essentially points "up" to the needs of the enterprise, directory data points "down" to encryption and control of system resources. Directories are critically important to employing both "front door" and "back door" security. The LDAP standard for directories is a fundamental element of Grid Computing, the next generation Internet/supercomputer standard being developed with major sponsorship by NSF and IBM.