From Zero to End-to-End Crypto in a Half Hour: Coding in Java with E3DB

Recently, we announced a developer preview of our End-to-End Encrypted Database, E3DB. In this post, we give some step-by-step instructions for developers to try out the example code.

First a caveat: Don’t store anything you want to keep (yet!). This is a developer preview, and we might have to delete the database periodically. Please do try it out as an experiment, and over time, we will build it into a durable encrypted storage solution!

When we do a real release, you can use Project E3DB for any type of data storage.

  • Do you collect data about your users? You should be storing it encrypted at rest so the bad guys can’t get it. You really should already be doing this.
  • Do you share data with third parties? Instead of sending them your data in the clear, make them read it encrypted at the source. It lets you stay in control and doesn’t encourage them to store their own plain text copy.
  • Do you want your users to communicate end-to-end encrypted? Each party can have their own key and send JSON blobs to each-other without you or us being able to decrypt it. Sweet.
  • Do you want to segregate data among your servers? Ever heard of the Principle of Least Privilege? Project E3DB lets you create a JSON object store where only the services that need access to the data will get the keys to the data. That means that if a bad guy breaks into one part of your system, they don’t get everything!

In this example, we will share end-to-end encrypted feedback with Tozny CEO Isaac Potoczny-Jones.

Clone and Build

Grab the example from our Github Repo:

git clone git@github.com:tozny/e3db.git
cd e3db
./sbt "run register"   # Invokes the CLI with the register command 
./sbt examples/compile # Compiles examples
./sbt examples/run     # Runs examples

Import into IntelliJ IDEA (Optional)

This step is optional, but helpful if you use the IntelliJ IDE.

  1. Make sure the scala and SBT plugins are installed.
  2. File -> New -> Project from Existing Sources
  3. Import project from external model -> SBT -> Next -> Finish
  4. Reveal and view the folder: e3db -> examples -> src -> main … -> ListRecords
  5. View build.sbt inside examples
  6. It should say “build.sbt is SBT build file” – click “import project”
  7. Right-click on ListRecords -> Run ListRecords.main()
  8. You should see some output reflecting anything you’ve already put into E3DB.

Edit the code

Open up the JavaDocs, which describe the SDK functions and open up ListRecords.java where you’ll find examples for writing records, reading records, and sharing by type.

Creating an E3DB Client

Each example class begins by creating a Client object via the class HttpE3DBClientBuilder. Using a builder-style interface makes it easy to pass in the necessary configuration settings and credentials needed to set up the client.

Client client = new HttpE3DBClientBuilder()
 .setClientId(clientId)
 .setApiKeyId(apiKeyId)
 .setApiSecret(apiSecret)
 .setKeyManager(keyManager)
 .build();

In the example code, parameters like clientId come from the config file or command-line arguments. In a production system, they would come from a secure credential storage system, or some other location.

Writing Records

Now that the client is configured, we can write a record into E3DB. It will be encrypted automatically and uploaded to the database.

  • This starts with constructing metadata about the record itself; the writer ID and client ID, as well as the content type.
  • The “feedback” content type is what we use for sharing end-to-end encrypted feedback about E3DB itself 🙂
  • The map from String to String is the set of name/value pairs that are converted to JSON. The values are encrypted and written to E3DB, all transparently.
Meta writeMeta = new Meta(clientId, clientId, "feedback");
HashMap <String, String> map = new HashMap();
map.put("comment", "Hello World! I successfully ran the example file.");
Record nameRecord = new Record(writeMeta, map);
UUID newWriteId = client.writeRecord(nameRecord);
System.out.println("Feedback Created: " + newWriteId);

If you want to experiment, here’s a good place to do it. Add more elements to the map, read in data from STDIN or from some configuration file, or create something fun like a shared TODO list or password manager!

Reading Records

In the above example, we received a record ID after writing the map to E3DB. Using this record ID, we can read the map back out. It gets downloaded from E3DB, transparently decrypted, and displayed on the command line.

Record feedbackRecord2 = client.readRecord(feedbackRecordId).get();
System.out.println ("Read the comment: " + feedbackRecord2.data.get("comment"));

There’s also a function “client.readRawRecord(id)” that fetches, but does not decrypt the data, so you can see the encoded ciphertext if you like.

Sharing Records

Currently, sharing is facilitated by content type. Our sharing model currently allows the other party to read all records of the selected content type; we may add more flexible sharing model in the near future. When you share data with another party, some crypto happens in the client to allow that party to read it, but without any other party having access, including us.

In the following example, we share the record that we created above with the UUID of Tozny’s CEO Isaac.

UUID ipjId = UUID.fromString ("166ed61a-3a56-4fe6-980f-2850aa82ea25");
client.authorizeReader(clientId, ipjId, "feedback");
PolicyRequest shareReq = new PolicyRequest(clientId,
 clientId,
 ipjId,
 Policy.allow(Policy.READ),
 "feedback"
);
client.setPolicy(shareReq);

Listing Records

It is simple to list records visible to the client by calling listRecords.

for (Meta meta : client.listRecords(100, 0)) {
 System.out.printf("%-40s %s\n", meta.record_id, meta.type);
}

For each record returned by listRecords, we receive an instance of Meta, which contains the following meta-information about each record in the data store. The record_id is particularly interesting, because that’s what you use for readRecord. For now, write and user IDs are more-or-less the same, but that will change in future versions, where you can read and write data about other users.

public class Meta {
 public final UUID record_id;
 public final UUID writer_id;
 public final UUID user_id;
 public final String type;
 public final Date created;
 public final Date last_modified;
}

Wrapping Up

That’s it! With these building blocks, you can create and store any type of data securely by encrypting it in the client, and sharing it with authorized parties. Authorized parties can be other users, your own web service, or a third party. If you’re curious about how the crypto works, we have a bit of information on our blog, and more coming soon!