Before we can discuss DB file management, we need to describe checkpoints. When databases are modified (that is, a transaction is committed), the modifications are recorded in DB's logs, but they are not necessarily reflected in the actual database files on disk.

This means that as time goes on, increasingly more data is contained in your log files that is not contained in your data files. As a result, you must keep more log files around than you might actually need. Also, any recovery run from your log files will take increasingly longer amounts of time, because there is more data in the log files that must be reflected back into the data files during the recovery process.

You can reduce these problems by periodically running a checkpoint against your environment. The checkpoint:

There are several ways to run a checkpoint. One way is to use the db_checkpoint command line utility. (Note, however, that this command line utility cannot be used if your environment was opened using DB_PRIVATE.)

You can also run a thread that periodically checkpoints your environment for you by calling the DB_ENV->txn_checkpoint() method.

Note that you can prevent a checkpoint from occurring unless more than a specified amount of log data has been written since the last checkpoint. You can also prevent the checkpoint from running unless more than a specified amount of time has occurred since the last checkpoint. These conditions are particularly interesting if you have multiple threads or processes running checkpoints.

For configuration information, see the DB_ENV->txn_checkpoint() API reference page.

Note that running checkpoints can be quite expensive. DB must flush every dirty page to the backing database files. On the other hand, if you do not run checkpoints often enough, your recovery time can be unnecessarily long and you may be using more disk space than you really need. Also, you cannot remove log files until a checkpoint is run. Therefore, deciding how frequently to run a checkpoint is one of the most common tuning activity for DB applications.

For example, to run a checkpoint from a separate thread of control:

#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>
#include "db.h"     

void *checkpoint_thread(void *);

int
main(void)
{
    int ret;
    u_int32_t env_flags;
    DB_ENV *envp;
    const char *db_home_dir = "/tmp/myEnvironment";
    pthread_t ptid;

    envp = NULL;

    /* Open the environment */
    ret = db_env_create(&envp, 0);
    if (ret != 0) {
        fprintf(stderr, "Error creating environment handle: %s\n",
            db_strerror(ret));
        return (EXIT_FAILURE);
    }

    env_flags = DB_CREATE     |  /* If the environment does not
                                  * exist, create it. */
                DB_INIT_LOCK  |  /* Initialize locking */
                DB_INIT_LOG   |  /* Initialize logging */
                DB_INIT_MPOOL |  /* Initialize the cache */
                DB_THREAD     |  /* Free-thread the env handle. */
                DB_INIT_TXN;     /* Initialize transactions */

    /* Open the environment. */
    ret = envp->open(envp, db_home_dir, env_flags, 0);
    if (ret != 0) {
        fprintf(stderr, "Error opening environment: %s\n",
            db_strerror(ret));
        goto err;
    }


    /* Start a checkpoint thread. */ 
    if ((ret = pthread_create( 
        &ptid, NULL, checkpoint_thread, (void *)envp)) != 0) { 
            fprintf(stderr, 
                "txnapp: failed spawning checkpoint thread: %s\n", 
                strerror(ret)); 
            goto err;
    }

    /* 
     * All other threads and application shutdown code 
     * omitted for brevity. 
     */

    ...
}


void * 
checkpoint_thread(void *arg) { 
    DB_ENV *dbenv; 
    int ret;

    dbenv = arg;

    /* Checkpoint once a minute. */ 
    for (;; sleep(60)) 
        if ((ret = dbenv->txn_checkpoint(dbenv, 0, 0, 0)) != 0) { 
            dbenv->err(dbenv, ret, "checkpoint thread"); 
            exit (1); 
        }

    /* NOTREACHED */ 
}