Re: Module Proposal for 3.0: Déjà Dup Backup Tool

[Paul Cutler <pcutler gnome org>]

On Sat, 2010-02-13 at 17:10 -0500, Michael Terry wrote:
> I'm not sure if this is too early, but it's been 6 months since
> modules proposals opened for 2.30, so I'm guessing it's an appropriate
> time for 3.0 modules?
> 
> I think GNOME could use a backup program.  There are some out there
> already, but having a GNOME-blessed one would focus development
> effort.
> 
> = What's my proposal? =
> 
> First, I'm specifically proposing my own pet project Déjà Dup (DD) for
> inclusion in the Desktop release set.  DD is only one of many backup
> programs out there. I think it's pretty sweet but would also like to
> open the floor for comparisons to or inclusion of other programs.
> Maybe mine isn't the best fit and overtures to other maintainers would
> be appropriate.
> 
> Déjà Dup's story is that it only tries to be a data backup program for
> disaster recovery. But aims to be the best little disaster recovery
> program there is.  It's main website is
> https://launchpad.net/deja-dup, and there is a mission statement for
> more details of why it exists (http://live.gnome.org/DejaDup/Mission).
> 
> It is designed to do the "right thing" without getting in the user's
> way.  It encourages backing up to a cloud service. It encourages
> backing up on an automatic schedule. It is designed to target the
> not-very-computer-literate. It prefers to backup too much rather than
> too little. It integrates well into GNOME.
> 
> It's also basically a glorified frontend for the command-line backup
> program duplicity.
> 
> Next, I will talk a little bit about where I see backups fitting into
> GNOME and whether it needs such a program at all.
> 
> == User Trust ==
> 
> If the user doesn't trust their backup system, what's the point? A
> backup system is only as strong as its restore, but by the time you
> find out that your system fails the restore test, it's usually too
> late.
> 
> Backups are tricky. We're responsible for user's data and they will be
> very mad if we lose it. Maybe GNOME doesn't want to get into that
> business, though one can always point to the boilerplate disclaimer of
> fitness for any particular task. :)
> 
> Déjà Dup is a wrapper around the already-proven command line tool
> duplicity.  (Although, Duplicity is a program under active
> development.  So it's not necessarily a rock-solid
> hasn't-needed-fixes-for-years tool.)  DD has a suite of automated GUI
> tests. So I trust it (and use it). But these facts aren't user
> visible.
> 
> I suspect user trust is a matter of time and usage. Unfortunately,
> Déjà Dup hasn't been around very long (about a year and a half now).
> 
> == Why Not Just Live in the Cloud? ==
> 
> More and more of our lives are being stored remotely in Internet
> services and clouds (flickr, facebook, Ubuntu One, rhapsody). This
> seems to reduce the need for a backup of your data, if it all lives in
> the cloud.
> 
> However, I don't think backup can ever be quite replaced. Unless a
> cloud system is specifically designed to hold your data, encrypted,
> with snapshots, with promises of availability, no-data-loss, and
> integrated into your desktop, there is still a strong use case for
> your own backup.
> 
>     * Most cloud services only keep track of your current files. If
> you delete a file and later realize you need it a month ago, you're
> out of luck.
>     * Cloud services tend to be fragmented (different services for
> different data). You don't have easy access to all your data from one
> place.
>     * There will likely be data that you don't trust the cloud to hold.
>     * You often only put a 'weaker' version of the data in the cloud,
> but keep the pristine version locally (e.g. photo upload to flickr).
>     * You may have large amounts of data that aren't appropriate for upload.
> 
> Of course, there's no reason you can't store your backup files in the cloud.
> 
> I look forward to the sci-fi future where everyone uses thin clients
> into server farms that offer transparent data safety and duplication.
> I won't stand in that future's way for love of my own code. But in the
> meantime, I think a backup program is a useful offering.
> 
> == What Does Backing Up Give Me? ==
> 
> There are already different solutions for the basic thrust of what
> backups do (data protection).
> 
>     * Rollback to previous versions of files with btrfs or some
> version of my-filesystem-is-a-vcs
>     * Sync local files to the cloud (like Ubuntu One)
> 
> But none of those are able to meet all disaster recovery needs (of
> three main sorts):
> 
>     * Revert to old version of file.
>     * Recover a small number of lost (accidentally deleted) files.
>     * Total system failure.
> 
> What many backup systems (and Déjà Dup specifically) provide are:
> 
>     * Rollback to previous versions
>     * Automatic backups (i.e. no-intervention after initial setup --
> if you have to do it manually, you're not doing it enough)
>     * Incremental backups (space saving)
>     * Encryption (so you can backup to non-trusted locations)
>     * Ease of off-site backup (for really bad disasters like floods,
> fires, or mass theft)
> 
> = Déjà Dup Details =
> 
> OK, hopefully I've convinced ya'll that a backup program is a useful
> offering.  Here's more information about my specific proposal.
> 
> == Dependencies of DD ==
> 
>     * duplicity (>= 0.5.03 but newer versions enable more DD features)
>     * python-boto (for Amazon S3 support)
>     * valac (generally the latest version, but it's only needed for
> building from tip)
> 
> These would need to be made external dependency modules?  (Except for
> valac, which I believe is already one).
> 
> == Maintainership and Community ==
> 
> Despite my efforts to entice more developers, it's basically just me.
> Only one other person has submitted patches that have been included.
> I have several users that help test the unstable branch and report
> useful bugs.
> 
> I'm biased, but I think I've been very responsive and active during
> Déjà Dup's lifetime. There are frequent releases and regular
> bug-fix-only and translation maintenance releases. I'm comfortable
> with the amount of maintainer work that GNOME seems to require (point
> releases leading up to a 6-month release).  In fact, for the past six
> months, I've been releasing versions of the unstable branch along with
> GNOME's schedule.
> 
> However, I suspect that inclusion into GNOME would (by virtue of a
> massive amount of new users) create lots more bug activity and patch
> submissions. This would create much more work for me (or at least a
> sense of obligation to do more work).
> 
> Frankly, I'm content with my current DD workload and don't want to
> increase it.  So I would probably scale back my feature work as more
> maintainer work cropped up.  How OK is GNOME with modules that don't
> add many/any features ever release?  I know there is a "Progress on a
> regular basis" requirement, but I'm not sure how that's scaled for
> developer team size.
> 
> Ideally I'd get a co-maintainer or some frequent developers? I'd even
> be happier if someone else wanted to maintain it and I would just be a
> developer on the project.  I'm curious about other maintainers'
> experiences about how GNOME-inclusion leads to more developers or
> maintainers.
> 
> == Resources ==
> 
> None of GNOME ftp, git, and bugzilla are used currently. Launchpad
> serves all project hosting needs right now. I'm fine switching to
> GNOME hosting, but am attached to Launchpad and wouldn't want to
> switch until approved.
> 
> Uses GNOME wiki already.  If adopted, I would use bugzilla, git, ftp,
> probably a mailing list?  I dunno, the normal new-project set of
> resources.
> 
> == Adoption ==
> 
> There are some number of active users, but obviously actual statistics
> are hard to come by.  There was an small article in Linux Format
> magazine about it, and a little press on the Internet.
> 
> Currently packaged in Debian and Ubuntu. No RPM distros yet package
> Déjà Dup (there is an open bug with a prospective spec file for
> Fedora). Duplicity is basically packaged everywhere.
> 
> == License ==
> 
> DD is GPLv3+, no copyright assignment
> Duplicity is GPLv2+, no copyright assignment
> 
> == GNOME Community Interaction ==
> 
> No particular overtures have yet been made. I'm excited to work with
> the various GNOME teams (documentation, i18n, usability, etc). I'm a
> GNOME contributor from way back and am familiar with the GNOME
> community on a superficial level.
> 
> == 3.0 Readiness ==
> 
> No deprecated libraries are used.
> 
> I'm not sure how/if it would fit into the current 3.0 plan, but backup
> is a missing hole in the market so to speak, so I suppose it fits in
> by default?
> 
> == GNOMEness ==
> 
>     * GTK
>     * Follows HIG
>     * Integrates into nautilus via an extension
>     * Uses session dbus 'locking'
>     * Uses NetworkManager dbus API to watch network connection
>     * Uses gnome-keyring to store passwords
>     * Uses gio/gvfs
>     * Keeps up with GNOME best practices (like X-GNOME-Fullname and the like)
>     * Written in vala
>     * Uses gnome-common and gnome-doc-utils
> 
> == Roadmap ==
> 
> I have several features in mind: Being able to browse your backup, a
> 'restore missing files' wizard, restoring a user's installed packages,
> optical (DVD/CD) backup, better backup management, etc.
> 
> DD is *not* feature complete.  But what is there should be reasonably
> polished.  I've historically focused on good experiences over
> features.
> 
> = Testing =
> 
> If you use Ubuntu, you're in luck.  Various versions are in Jaunty,
> Karmic, and Lucid.  I also make a Karmic PPA available of the latest
> development version:
> https://launchpad.net/~deja-dup-team/+archive/testing
> 
> Else, download the latest tarball and build like normal (you won't
> need valac for this):  https://launchpad.net/deja-dup/+download
> 
> I would add it to jhbuild's proposed list, but the gnome 3.0 moduleset
> doesn't exist yet.
> 
> 
> 
> Thoughts?
> -mt
> _______________________________________________
> desktop-devel-list mailing list
> desktop-devel-list gnome org
> http://mail.gnome.org/mailman/listinfo/desktop-devel-list

>From a marketing perspective, the inclusion or addition of Deja Dup as a
backup utility is a positive.  We had a brief conversation about it at
the Marketing Hackfest this week and backup as a utility is something
our users need and has value in the desktop.

Paul