Exchange Storage ArchitectureExchange servers store data in two files: an .edb file and an .stm file. Together, the .edb file and the .stm file form an Exchange store repository. For example, the default mailbox store on an Exchange server uses files named Priv1.edb and Priv1.stm. The default public folder store uses the files Pub1.edb and Pub1.stm. The .edb file contains many tables that hold metadata for all e-mail messages and other items in the Exchange store, in addition to the contents of MAPI messages. The .edb file is an ESE database, and because it is used primarily to store MAPI messages and attachments, it is also referred to as the MAPI-based database. The .stm file, in contrast, stores native Internet content. Because Internet content is written in native format, there is no need to convert messages and other items to Exchange format (as in Exchange 5.5 and earlier). The .stm file is also an ESE database, referred to as the streaming database. The .edb and .stm files function as a pair, and the database signature (a 32-bit random number combined with the time that the database was created) is stored as a header in both files. The internal schema for the .stm pages is stored in the .edb file. Note You can rename the .edb and .stm databases and move them to different directories in Exchange System Manager. Because the .edb and .stm files together create a complete Exchange store repository, you should keep them together and assign them a common name with different extensions (that is, .edb and .stm). Exchange Server 2003 uses transactions to control changes in storage groups. These transactions are recorded in a transaction log, similar to the way transactions are stored in traditional databases. Changes are committed or rolled back based on the success of the transaction. If there is a failure, you use transaction logs (together with the database files and, in some cases, the checkpoint file) to restore a database. The facility that manages transactions is the Microsoft Exchange Information Store service (Store.exe). Any uncommitted transaction log entries are also considered part of a current Exchange database, as illustrated in the following figure. Current Exchange Server 2003 database The following two types of databases are available in Exchange Server 2003:
The following figure illustrates the internal Exchange store architecture. The Microsoft Exchange Information Store service (Store.exe) uses Extensible Storage Engine (ESE) to access the database files in the file system, and provides access to the data through various interfaces, such as MAPIsvr, ExPOP, ExIMAP, ExSMTP, and ExOLEDB. Client application and application programming interfaces, such as Collaboration Data Objects for Exchange (CDOEX), can use these interfaces or communicate with the messaging database (MDB) module. Exchange store architecture Storage GroupsEach storage group is made up of a set of log files and auxiliary files (internal temporary databases, the checkpoint file, and reserve logs) for all the databases (.edb files, .stm files) in the storage group. Exchange Server 2003 supports multiple storage groups and multiple databases in each storage group. In Exchange Server 2003, a single server supports up to four storage groups and a single storage group supports up to five databases. Support for multiple databases enables you to distribute numerous mailboxes and public folders across numerous, smaller databases, thus making database management easier. Exchange 2000 Server and Exchange Server 2003 can support up to 20 mailbox and public folder databases on a single server. Storage Group ArchitectureAs illustrated in the following figure, all storage groups are hosted from the same Store.exe process. Each storage group is represented by an ESE instance. Storage group architecture Within each storage group, each .edb and .stm database pair represents a mailbox store or a public folder store. As shown in Figure 10.3, all mailbox and public folder stores in a particular storage group share a common set of log files and other system files. These files enable transaction-oriented processing. The log files and other system files in each storage group have the following purposes:
Storage Group Attributes in Active DirectoryYou can determine the path to a storage group's transaction log file and the log file's name in Exchange System Manager. Right-click the desired storage group, select Properties, and from the General tab, look at the information in the Transaction Log Location and the Log File Prefix fields. Using the Browse buttons, you can move the transaction log and system files to a new location, such as a separate physical drive. The configuration settings for a storage group are stored in Active Directory. If you want to use ADSI Edit to locate the directory object for a storage group, you must open the configuration naming contacts, expand the services node, then CN=Microsoft Exchange, and then expand the Exchange organization object, administrative group, and server container. Underneath it, you can find a container named CN=InformationStore, which contains the storage groups, such as CN=First Storage Group. The object class for storage group objects is msExchStorageGroup. If you plan to use custom scripts to manage Exchange store resources, you can access msExchStorageGroup objects by using Active Directory Service Interfaces (ADSI). The following code example demonstrates how to access the default storage group on a server called SERVER01 in an Exchange organization called Contoso. It displays the current path to the transaction log files of that storage group. strStorageGroupDN = "CN=First Storage Group," _ & "CN=InformationStore," _ & "CN=SERVER01,CN=Servers," _ & "CN=First Administrative Group," _ & "CN=Administrative Groups," _ & "CN=Contoso,CN=Microsoft Exchange," _ & "CN=Services,CN=Configuration," _ & "DC=Contoso,DC=com" Set oStorageGroup = GetObject("LDAP://" & strStorageGroupDN) MsgBox oStorageGroup.Get("msExchESEParamLogFilePath") The following are important Exchange attributes of msExchStorageGroup objects that you can use in custom scripts based on ADSI:
Storage Group Minimum Disk Space RequirementsEach storage group consumes about 50 MB of free disk space. The files listed above that are required by the storage group use a minimum of 11 MB of disk space. The minimum disk space for private and public stores is 5 MB and 8 MB, respectively. Although the total disk space used is about 24 MB, extra disk space is also needed for the actual creation of the storage group and for read and write operations. When working with storage groups, remember the following:
Exchange Store DatabasesExchange Server uses ESE as an embedded database engine that determines the structure of the databases and manages memory. The database engine caches the databases in memory by transferring four-kilobyte chunks of data (pages) in and out of memory. It updates the pages in memory and writes new or updated pages back to the disk. When requests come to the system, the database engine can buffer data in memory, so that it does not have to access the disk constantly. This makes the system more efficient, because writing to memory is approximately 200,000 times faster than writing to disk. When users make requests, the database engine starts loading the requests to memory and marks the pages as dirty. A dirty page is a page in memory that contains data. These dirty pages are later written to the Microsoft Exchange Information Store service databases on disk. Although caching data in memory is the fastest and most efficient way to process data, it means that while Exchange is running, the information on disk is never completely up-to-date. The latest version of the database is in memory, and because many changes in memory are not on disk yet, the database and memory are not synchronized. If there are any dirty pages in memory that have not been transferred and written to disk, the databases are flagged as inconsistent. Exchange databases are synchronized only when all dirty pages in memory are transferred to disk. This happens when you properly shut down the Microsoft Exchange Information Store service. During the shutdown process, the Microsoft Exchange Information Store service flushes all pages to disk. MAPI Database FileThe Exchange Server 2003 MAPI database file contains the tables that hold the metadata for all e-mail messages, other objects in the database, and the contents of MAPI messages. Every folder displayed in Microsoft Office Outlook is a separate database table in the Exchange store. Every sort order used to view these folders is represented by a separate index on that table. The Store.exe process manages these sort orders. Messages from MAPI clients, such as Outlook, are stored in the MAPI database, just as they were stored in previous versions of Exchange Server. MAPI-based clients can then access these messages without conversion. However, if an Internet protocol-based client attempts to read a message in this database, the message is converted to the requested format. The traditional .edb file and its accompanying .stm file are a single unit. One of these files is of little use without the other file. It is important to understand that a single database in the Microsoft Exchange Server Information Store service contains two files, the .edb file and the .stm file. A record in the .edb file contains a column (of data type JET_coltypSLV) that references a list of pages in the streaming file that contains the raw data. Space usage (maximum of four kilobytes of page numbers) and checksum data for the data in the streaming file is stored in the .edb file. Streaming Database FileExchange Server 5.5 and earlier store messages in message database encapsulated format (MDBEF). This is the native format for Outlook clients. When a non-MAPI client requests a message, the Microsoft Exchange Information Store service converts the contents from MDBEF to the appropriate format, based on what the client requests. This conversion consumes processor bandwidth and slows server performance. Later versions of ESE enable Internet messaging clients to store raw data in native format. The repository for this raw data is referred to as the streaming database, or simply the streaming file. The streaming file has no balanced tree (B-tree) overhead. Instead, it contains two four-kilobyte pages of header information and then raw data in four-kilobyte pages. This flat data structure is designed for binary large objects (BLOBs) of data that are unlikely to need content conversion and that can be received and transmitted very quickly. Property PromotionProperty promotion determines where data is stored in an ESE database and is therefore an important concept to understand. The Microsoft Exchange Information Store service supports the property promotion of data held in the .stm file to the .edb file. Property promotion enables folder views and indexes to be maintained efficiently. For example, a message streamed to the .stm file has its properties, such as sender, subject, and date sent and received, promoted to the records representing the message in the .edb file. When a MAPI client, such as Microsoft Outlook, submits a message to the Microsoft Exchange Information Store service, the contents of that message are stored in the .edb file. If a non-MAPI client opens the message, the Microsoft Exchange Information Store service does an immediate conversion of the MAPI content to Internet format by performing some of the conversion and calling IMAIL, which in turn calls RTFHTML, to complete the conversion. None of this conversion is persistent, meaning that data is not moved out of the .edb file and written to the .stm file. If an Internet client submits a message to the Microsoft Exchange Information Store service, the contents of that message are stored in the .stm file. Certain headers from the Internet message are duplicated to the .edb file, so the Microsoft Exchange Information Store service can find the message. This is referred to as a state 0 conversion. If any client asks for a property, such as PR_Subject, or one of its many aliases, then the Microsoft Exchange Information Store service promotes all of the Internet message's header information to Properties. This is referred to as a state 1 conversion. If any client asks for attachment information, then the Microsoft Exchange Information Store service creates a near duplicate (in MAPI form) of the Internet message. At first, the message is still in the .stm file. However, much of the data needed for MAPI access is in the .edb file. If a client alters the message in a way that changes the Multipurpose Internet Mail Extensions (MIME), then the .stm file version of the message is discarded and the .edb file of the message is preserved. This is referred to as a state 2 conversion. Regardless of how a message is submitted to the Microsoft Exchange Information Store service, if Exchange Server receives Internet content that includes Application/ms-tnef content, the message initially goes to the .stm file, but it is then immediately decoded and moved to the .edb file. The same applies to messages with a winmail.dat attachment, encoded using UUEncode. Transport neutral encapsulation format (TNEF) and Winmail.dat are encapsulation methods for MAPI messages to preserve MAPI properties on transports that do not support MAPI. Therefore, the general principal that MAPI messages reside in the .edb file and Internet messages reside in the .stm file is correct. The current functionality has the TNEF decoded before any one of the MAPI properties are read. |