A common server infrastructure, providing state replication services for collaborative web applications.

Project Overview

Laasie is a data management system designed specifically for collaborative web applications. Unlike normal desktop applications, collaborative applications live in the cloud. Multiple users can simultaneously interact with a collaborative application by loading client front-ends that concurrently display, and update the application's user interface.

Collaborative applications are pervasive in the modern world -- examples range from Google Docs/Office 365, to Dropbox, Evernote, and even applications like Twitter or Facebook can be thought of in these terms.

Laasie streamlines the development process for collaborative applications by providing a common server infrastructure for state replication, communication, and persistence, freeing developers to focus their efforts on the client, where their innovation is truly needed.

What Laasie Provides

Persistence

Laasie exposes a simple, unstructured, JSON-style abstraction for storing application state. Data written to the log is persisted, even through client and server failures.

Replication

Laasie ensures that each client sees an eventually consistent view of the application state. Users interact with the same data in near real-time.

Recoverability

A client that disconnects from the network, goes to sleep, or otherwise breaks its connection to the server, does not need to be restarted. Laasie handles all of the messiness of bringing the client up-to-date with everything that happened while it was disconnected.

Undo

Laasie keeps state histories for as long as you want. If someone makes a bad modification by mistake, or worse, maliciously... Laasie will soon support efficient undo functionality.

A Little History

Laasie is a next-generation version of the ICON collaborative application data management system developed as part of the Hugin Mapper. In addition to being developed from the ground up as a standalone server infrastructure, Laasie incorporates an advanced technique for storing application state called a monadic log. Monadic logs give the Laasie infrastructure unprecedented flexibility in balancing server resource utilization, and will eventually lead to improved coordination techniques.

The Team

  • Oliver Kennedy
  • Luke Ziarek
  • Sumit Agarwal
  • Daniel Bellinger
  • Ankur Upadhyay

Engineering Challenges

Proxy

The Laasie proxy is a system component that mediates between the client application and the Laasie server. It is typically implemented as a client side library, and is responsible for

  • Maintaining a local replica of the application state as a JSON object
  • Managing client callbacks for notification of updates to the application state
  • Forwarding client state updates to the server
  • Managing the push/pull interface to the server infrastructure.
  • A secondary purpose for the proxy library is to provide datastructure support, abstracting the JSON application state representation into language-specific constructs (e.g., Lists, Table Views, etc..)

We currently have proxies implemented in the following languages

  • Cappuccino

Extend Language Support

We are looking for people to implement proxy functionality in other languages such as Java, Javascript, and Cocoa/ObjC, and to extend existing language support by adding new datastructures, and support for our new state mutation and query language BarQL. The primary goal of this task would be to replicate proxy functionality (maintaining the JSON object, applying updates, handling notifications, and providing supplemental datastructure and UI support functionality).

Local Storage

There are many situations where a client is forced to restart: closing the browser window, restarting for software updates, crashes, etc... The potentially high startup cost of state transfer can be avoided for many applications by using local storage to persist the client's state replica (e.g., HTML5 local storage for Cappuccino and/or Javascript). The primary challenge of this task is to build a local persistence mechanism that can keep its persisted state synchronized [b]efficiently[/b]. This may include re-implementing fragments of Laasie's monadic log within the host language, or using some form of checkpointing on the local state.

State/Storage

Persistent Logging/Crash Recovery

Laasie is presently implemented entirely in-memory. To be fully functional, updates must be persisted to disk. This task includes extending Laasie with a crash-resilient disk-resident update log, and mechanisms for reconstructing state. Extensions to this task include adding support for checkpointing for quicker recovery, garbage collection, and adding support for reverting prior operations -- even if relevant state is now resident only on-disk.

Blob Storage

As presently implemented, Laasie is ill suited for use with large bulk data objects. Our current plan to deal with these is to implement an external blob-store. This blob store would integrate with the core Laasie instance and provide storage functionality for large, potentially changing objects, in a manner analogous to LBFS.

Workload Tests / Demo Apps

A crucial challenge for Laasie is developing a thorough understanding of what developers need from Laasie, how developers might use Laasie, and where our design, development, and optimization efforts are required. Central to this challenge are real-world applications actually built on Laasie.

Client Software

We need applications built on top of Laasie to help us exercise functionality, develop APIs, and identify performance bottlenecks in our system. Examples of applications that might be built are:

  • A Chat Client
  • Calendaring Software
  • A Text Editor

Protocol Mapping

A second, related source of workloads is an idea that will be eventually developed on the research side: establishing mappings between Laasie and legacy protocols. For example, one might consider an interface to connect existing IRC clients to Laasie, or Laasie to existing IRC servers. Similar mappings can be built for other popular protocols (e.g., IMAP, POP, CalDav, Jabber). The goal of this task is to identify

Our long term strategy is to create a tool to simplify the creation of these mappings. See below under research challenges.

Research Challenges

Performance Studies

Our preliminary tests suggest that for collaborative application workloads, Laasie produces a substantial performance improvement. However, we believe that a thorough benchmarking study of existing data management technologies, as well as of Laasie, under a variety of collaborative application workloads will lead to new and advanced techniques for collaborative application data management. This first requires surveying existing collaborative applications to obtain a thorough understanding of their data access patterns, and expanding these workloads into a concrete database/noSQL system benchmark.

Monadic Log Views

Storing updates in functional form allows Laasie to express a wide variety of semantics. This includes the ability to perform introspective updates. Queries can be performed on data that matches certain patterns. For example, a client requiring tighter integrity constraints on a dataset than the application normally permits can place a request for all updates that do not violate the integrity constraint. A client can request to ignore all updates performed by one or more "hostile" users.

Incremental Maintenance

BarQL

Datastructure Mappings