Best Practices: Arrays in Firebase

April 28, 2014

Storing and iterating arrays comprises a good portion of the Firebase questions on StackOverflow. This article will explain Firebase’s array support, the ins-and-outs of arrays in distributed data, and some common techniques for dealing with array-like data.

First, let’s get a handle on the current array support.

Firebase’s Array Support

Firebase has no native support for arrays. If you store an array, it really gets stored as an “object” with integers as the key names.

// we send this
['hello', 'world']
// Firebase stores this
{0: 'hello', 1: 'world'}

However, to help people that are storing arrays in Firebase, when you call .val() or use the REST api to read data, if the data looks like an array, Firebase will render it as an array.

In particular, if all of the keys are integers, and more than half of the keys between 0 and the maximum key in the object have non-empty values, then Firebase will render it as an array. This latter part is important to keep in mind.

// we send this
['a', 'b', 'c', 'd', 'e']
// Firebase stores this
{0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e'}

// since the keys are numeric and sequential,
// if we query the data, we get this
['a', 'b', 'c', 'd', 'e']

// however, if we then delete a, b, and d,
// they are no longer mostly sequential, so
// we do not get back an array
{2: 'c', 4: 'e'}

You can’t currently change or prevent this behavior. Hopefully understanding it will make it easier to see what one can and can’t do when storing array-like data.

So why not just store arrays instead of objects?

Arrays are Evil

I bet that got your attention. Actually, arrays are quite handy. But for distributed data, they aren’t reliable because they lack a unique, permanent way to access each record.

Consider the case where three clients attempt to modify an array on a remote service like Firebase:

Initial data: [ {name: 'foo', counter: 1}, {name: 'bar', counter: 1}, {name: 'baz', counter: 1} ]
Client A moves the “bar” record to position 0
Client B attempts to delete the “bar” record at position 1
Client C tries to update bar’s counter with something like data[1].counter++

The end result is that record ‘foo’ is deleted, and the counter for record ‘baz’ is incremented, while the now incorrect ‘bar’ sits at position 0. None of the operations worked quite as intended.

If the data were instead stored as an object, keyed by unique ids, the scenario would look more like this:

Initial data: {foo: {counter: 1}, bar: {counter: 1}, baz: {counter: 1}};
Client B deletes “bar” from the object
Client C attempts to update bar’s counter but sees that it does not exist

Now the operations resolve in a sensible manner, despite concurrent updates from multiple clients. This isn’t possible with an array, where the indices used to reference each record are fluid and change over time.

However, we couldn’t just pick up the bar record and move it to the front of the list. What we need is a convenient way to create unique ids, such as a hash or object’s keys, while still having a defined ordering for the records.

In Firebase, this is done with push ids and priorities.

Using Push to Create Unique IDs

Firebase’s solution to the problem of unique identification is the push method (note that this is named childByAutoId in iOS).

Like JavaScript’s push method, this appends new records to the end of our collection. It does this by assigning a permanent, unique id based on the current timestamp (offset to match server time). This means that each record is naturally sorted at the end and that they maintain a chronological order.

var ref = new Firebase(URL_TO_DATA);
// this new, empty ref only exists locally
var newChildRef = ref.push();
// we can get its id using key()
console.log('my new shiny id is '+newChildRef.key());
// now it is appended at the end of data at the server
newChildRef.set({foo: 'bar'});

Using Priorities to Order Records

Firebase provides a set of tools for ordering data at the server level. By using setPriority or setWithPriority, it’s possible to specify the order in which records are stored and retrieved. A classic example of this is a leaderboard, where results are stacked by highest score.

var scoresRef = new Firebase(URL_TO_SCORES);
var score = 10000000000; // enough to put me in the #1 position!

// we sort the scores in descending order, so use a negative score for the priority ordering
scoresRef.child('kato').setWithPriority( score, -score );

// retrieve the top 10 scores
scoresRef.limitToLast(10).once('value', function(snap) {
   var i = 0;
   snap.forEach(function(userSnap) {
      console.log('user %s is in position %d with %d points', snap.key(), i++, snap.val());
   });
});

Good Use Cases for Storing Data in Arrays

But what about client-side sorting, such as by table heading, and other ops that really just don’t work with objects? There are times when nothing but an array will do.

If all of the following are true, it’s okay to store the array in Firebase:

only one client is capable of writing to the data at a time
to remove keys, we save the entire array instead of using .remove()
we take extra care when referring to anything by array index (a mutable key)

 // initial data: [ {name: 'foo', counter: 1}, {name: 'bar', counter: 1}, {name: 'baz', counter: 1} ];
 var ref = new Firebase(URL_TO_LIST);

 // sync down from server
 var list = [];
 ref.on('value', function(snap) { list = snap.val(); });

 // time to remove 'bar'!
 // this is the correct way to change an array
 list.splice(1, 1);
 ref.set(list);

 // DO NOT DO THIS! It will eventually turn the array into an object
 // and may result in null values for deleted indices
 // ref.child('1').remove();

Arrays in Java

In Java, if we know that the data is array-like, it can be cast as a List:

Firebase julieRef = new Firebase("https://SampleChat.firebaseIO-demo.com/users/julie/");
julieRef.addValueEventListener(new ValueEventListener() {
   @Override
   public void onDataChange(DataSnapshot snapshot) {
       GenericTypeIndicator<List<String>> t = new GenericTypeIndicator?<List<String>>() {};
       List messages = snapshot.getValue(t);
       if( messages === null ) {
          System.out.println('No messages');
       }
       else {
          System.out.println("The first message is: " + messages.get(0) );
       }
   }

   // onCancelled...
});

Arrays in iOS

In Objective-C, array-like data is easy enough to iterate. However, some use cases, such as Table View, require the use of NSArray. It can be obtained with a simple type cast:

Firebase *julieRef = [[Firebase alloc] initWithUrl:@"https://SampleChat.firebaseIO-demo.com/users/julie"];
[julieRef observeEventType:FEventTypeValue withBlock:^(FDataSnapshot *snapshot) {
  if(snapshot.value == [NSNull null]) {
    NSLog(@"No messages");
  } else {
    NSArray *messages = [snapshot.value];
    NSString *firstMessage = [messages objectAtIndex:0];
    NSLog(@"First message is: %@", firstMessage);
  }
}];

More About Arrays

What if we want shiny arrays, and still want to have all the power of real-time collaboration? Well Firebase is about building amazing real-time apps so surely we can have both!
Check out Synchronized Arrays in Firebase to learn more about this advanced technique.

Find this article useful or have additional questions? Leave a comment below or post a message on the mailing list!

The Firebase Blog