- Thomas' Webjar full of joy -
| Author: | Thomas Langewouters |
|---|---|
| tags: | software |
| description: | A distributed file tracking system |
I have several computers, and find my files scattered all over PC's, disks and several locations.
You can find the last incarnation of the Ltr architecture at https://github.com/thouters/ltr.
I'm growing tired of having to power on machines to check if some of my stuff is on them, and having a crappy back-up policy.
I want file listings at hand of all my disks, when I'm offline.
I looked into distributed filesystems, version control systems and online services like Dropbox and Ubuntu One. All of them solve a different problem.
Over time I shaped my own vision on how some of these things could be implemented, and I named it litter.
These are the highlights:
Ltr uses a CouchDB for file metadata, instead of keeping it on disk along the instances of a volume (LtrBox). The Ltr client needs to access this database when synchronising the state between LtrBox-es or the database. A ".ltrcookie" is stored along with the files, containing the Volume name (databasename) and unique LtrBox identifier.
You can navigate your files using indirectly, using a CouchApp in on the metadata database.
Litter uses any directory/filesystem to store your files data, and ssh or direct access to move data between disks.
A LtrBox has a policy attribute regarding automatic copies of files to the volume instance. Suggesting it contain all files, a cherry-picked subset, or only the directory structure skeleton.
Filenames are matched against an 'ignore' glob list by ltr's filesystem interface, matching files are ignored.
Having several LtrBoxes of the same volume across multiple disks effectively replaces backups.
Putting a LtrBox on a portable disk with a special policy of x gigabytes of updates can be an efficient tool of provisioning an off-site computer with file changes without network connections.
A centralised master-copy is easily implemented. A shell account with LtrBox folder and CouchDB are very close to the normal usecase.
The master-copy LtrBox has a never-delete policy set, providing time-machine like functionality.
Keeping sha1sum's and checking once in a while is a measure against silent file corruption.
Imagine a LtrBox on your phone's memory card, containing a subset of your music files. Picking the music you want on it using a CouchApp running on the phone. Running a Ltr client on your laptop to update the memory card to reflect the state of the database.