Reduce index costs with map fields

This page describes how to use a map field to manage index settings for a group of subfields.

As a best practice, you should remove unused indexes to reduce storage costs and improve write performance. By default, Cloud Firestore builds a single-field index for each field in a document. You can control single-field indexing by defining index exemptions but with a maximum of 200 single-field index exemptions per database. It's possible to reach this limit before disabling all your unused single-field indexes.

You can avoid reaching the exemption limit by grouping document fields with the same index requirements under a map field. You can then apply an index exemption to the map field, and the same exemption applies to the map's subfields.

Solution: Use map fields to help manage indexes

Imagine an app that depends on a collection of game_event documents. Consider the following two data models:

Top-level document fields

Node.js

db.collection('game_events').doc().set({
   timestamp: Firestore.FieldValue.serverTimestamp(),
   user_id: 'huDIl8H88kFAFAdcHayf',
   team_id: '6Q5BhBESeTPk8LT0O59I',
   event_type: 'rare_item_drop',
   display_text: 'You found a rare item!',
});

Map field and subfields

In this data model, all the document fields become subfields of the details field:

Node.js

db.collection('game_events').doc().set({
  details: {
    timestamp: Firestore.FieldValue.serverTimestamp(),
    user_id: 'huDIl8H88kFAFAdcHayf',
    team_id: '6Q5BhBESeTPk8LT0O59I',
    event_type: 'rare_item_drop',
    display_text: 'You found a rare item!',
  }
});

Assume this app always queries game_event documents based on user_id and timestamp or team_id and timestamp. For example:

Node.js

let query_user_events = db.collection('game_events')
                          .where('details.user_id', '==', 'huDIl8H88kFAFAdcHayf')
                          .orderBy('details.timestamp');

let query_team_events = db.collection('game_events')
                          .where('details.team_id', '==', '6Q5BhBESeTPk8LT0O59I')
                          .orderBy('details.timestamp');

Notice the following about this app:

• The app depends on the composite indexes for details.user_id, timestamp and details.team_id, timestamp.
• The app does not use the single-field indexes for timestamp, user_id, team_id, event_type, or display_text.

Based on these index requirements, it's a good idea to disable the single-field indexes for timestamp, user_id, team_id, event_type, or display_text. Now, compare the exemptions required for the two data models.

Disabling indexes for top-level fields

To disable the single-field indexes in a top-level fields data model, you must define an exemption for each field. Your exemption count increases by five, and if you add a new field to your data model, you must define another exemption to disable its single-field index.

Disabling indexes for subfields

To disable the single-field indexes for a map and subfields data model, you can define a single exemption for the map field. An exemption on a map field applies the same indexing settings to the map's subfields. If you add a new subfield to the details field, the exemption automatically disables the new subfield's single-field index.

For example, using the Firebase CLI, add this index exemption to your firestore.indexes.json file to disable the single-field indexes for the game_events collection:

{
    "collectionGroup": "game_events",
    "fieldPath": "details",
    "indexes": []
},

Note: You can also add index exemptions through the console.

If you later require a single-field index for one of the subfields, you can override the map field's index setting with an exemption. An exemption on a subfield overrides that subfield's inherited index settings. For example:

{
    "collectionGroup": "game_events",
    "fieldPath": "details.event_type",
    "indexes": [
      {
        "order": "ASCENDING",
        "queryScope": "COLLECTION"
      },
    ]
},

When to use this approach

In the example above, the map and subfields approach reduced the number of exemptions from five to one. Imagine, however, a similar document data model with two-hundred fields. This approach reduces the number of exemptions from 200 to 1.

You should consider using a map field and subfields approach when your document data model contains multiple fields with unused single-field indexes. You should especially consider this approach for documents with many fields.