External file support is designed for efficient storage of large objects. An object is considered to be large if it is more than a third of the size of a page. Without external file support, large objects must be broken up into smaller pieces, and then reassembled and/or disassembled every time the record is read or updated. Berkeley DB external file support avoids this assembly/disassembly process by storing the large object in a special directory set aside for the purpose. The data itself is not kept in the database, nor is it placed into the in-memory cache.
External files can only be stored using the data portion of a key/data pair. They are supported only for Btree, Hash, and Heap databases, and only so long as the database is not configured for duplicate records, or duplicate sorted records. In addition, the DBT that you use to access the external file data cannot be configured as a partial DBT if you want to access the data using the external file's streaming interface (introduced below).
Note that if the environment is transactionally-protected, then all access to the external file is also transactionally protected.
External files cannot be used with in-memory-only databases.
In order to use Berkeley DB's external file support, you must set
the external file threshold to a non-zero positive value. (The
default value is 0, which means that external
files are not enabled). You set the external file threshold using
either
DB->set_ext_file_threshold()
or
DB_ENV->set_ext_file_threshold()
.
Once external files are enabled, there are two ways to create an external file record:
Configure the DBT used to access the external file data (that is, the DBT used for the data portion of the record) with the DB_DBT_EXT_FILE flag. This causes the data to be stored as an external file regardless of its size, so long as the database otherwise supports external files.
Alternatively, creating a data item with a size greater
than the external file threshold will cause that data item
to be stored as an external file. Of course, for this
method to work, the external file threshold must be greater
than 0.
external files may be accessed in the same way as other DBT data, so long as the data itself will fit into memory. More likely, you will find it necessary to use the external file streaming API to read and write external file data. You open an external file stream using the DBC->db_stream() method, close it with the DB_STREAM->close() method, write to it using the the DB_STREAM->write() method, and read it using the DB_STREAM->read() method.