Enable storage and retrieval of custom and intrinsic objects
Use case
Currently, a JavaScript Data Format file stores data in transactions that atomically correspond to a single update of the in-memory data graph. It does so persisting primitive value sets as they are and by serialising the set action of complex values using JSON.stringify()
.
This is perfectly fine for arrays and plain objects but does not work custom objects (class instances) or for intrinsic objects* like Date
instances.
Example:
const JSDB = require('@small-tech/jsdb')
class Person {
constructor (name = 'Jane Doe') {
this.name = name
}
introduceYourself () {
console.log(`Hello, I’m ${this.name}.`)
}
}
const db = JSDB.open('db')
// Initialise the people table if it doesn’t already exist.
if (!db.people) {
db.people = [
new Person('Aral'),
new Person('Laura')
]
}
The first time you run this, since the Person
instances are in memory as created (not as loaded from the database), you will be able to call the introduceYourself()
method:
db.people.where('name').is('Laura').getFirst().introduceYourself()
// Outputs: Hello, I’m Laura.
However, on subsequent runs, you will get an error:
TypeError: db.people.where(...).is(...).getFirst(...).introduceYourself is not a function
This is because the loaded-in objects are plain objects, not instances of the Person
class. They are currently stored as below:
_[0] = JSON.parse(`{"name":"Aral"}`);
_[1] = JSON.parse(`{"name":"Laura"}`);
Proposed solution
During the set
handler of the data proxy, we can check whether an object is a plain object (obj.constructor.name === 'Object'
) or a custom one (all others, except for arrays, etc. See notes on intrinsic objects at end) and write out the code to recreate it (if the class exists) when the table is loaded back in. The statements are written out in a single statement/transaction. If this is not possible via chaining, etc., it should be implemented as a IIFE:
e.g., for the example above, the relevant transactions in the table would be:
_[0] = Object.create(typeof Person === 'function' ? Person.prototype : {}, Object.getOwnPropertyDescriptors(JSON.parse(`{"name":"Aral"}`)));
_[1] = Object.create(typeof Person === 'function' ? Person.prototype : {}, Object.getOwnPropertyDescriptors(JSON.parse(`{"name":"Laura"}`)));
Update: Note, the above will not work with transactions due to the JSON serialisation of the own properties. Instead, we must create a bare instance first and then populate its properties recursively as we do with regular objects and arrays, etc.
Prerequisites
Other effects
I was considering implementing fast compaction for smaller tables (i.e., ideally under ~65MB, where string handling begins to slow down, or 1GB, the upper limit of string size) where the whole table would be compacted using a synchronous JSON.stringify()
into a single serialised JSON string as part of a single JSON.parse()
statement. This would provide an orders of magnitude speed increase in compaction for smaller tables over what we do now, which is to replay and persist the in-memory object graph.
However, if we implement support for custom objects, we won’t be able to implement this for obvious reasons.
- Note that some intrinsic objects, like Date, will require special casing.
e.g., We can detect and implement support for Date like this:
_[0] = new Date('<date string from (new Date()).toJSON()>'>
I need to do more research into other collections, etc., like Map
, Set
, TypedArray
, ArrayBuffer
, etc.