Metacity Proposal: Grouping Windows

[Ryan McDougall <ryan mcdougall telusplanet net>]

Proposal to Add Grouping to Metacity, and some Rationale

0. Abstract

Solution presented for solving problems associated with window
management by allowing users to categorically organize windows by
defining custom window groups, and performing window transformations on
the objects that they represent in "desktop space".

1. Problem specification and background

The Windows-Icons-Mouse-Pointer paradigm has given our computers
excellent spatial multitasking abilities. Unfortunately like the clutter
of paperwork that occasionally builds up on our real world desks, a
profusion of windows can mean a real nuisance or even hindrance to
usability due to manual window management. 

1.1. The problem in Linux/GNOME platform

1.1.1 Virtual desktops

The traditional Unix approach to window organization is multiple
desktops. This technique helps us to categorize our windows into
meaningful groups, and reduces many window management tasks into
clicking on a desktop pager. However the setup and management of those
desktops is entirely manual, and does not allow one to interact with the
desktop as if it was a whole object. More abstractly, the concept of
multiple "virtual" desktops may be more confusing to new users:

A random, non-scientific sample of 20 junior Computer Science students
at a major university running Ximian GNOME 1.4 on Sun workstations,
shows that only 3 users made use of more than one desktop. All were
running multiple tasks/windows (usually emacs, terminal, mozilla). When
5 students were asked why they weren't using the extra desktops, three
didn't know what that they existed, and two didn't consider it worth the
effort to maintain multiple desktops. This is purely anecdotal and
*unscientific*, but hopefully illustrative.

Currently virtual desktops are very handy, but provide an inconsistent
spatial representation, and lack fist class status as objects in that
space. They are passive bins which hold windows, and I think we can do
better.

1.1.2 Document-Tabbed UIs

Recent UI changes in GNOME, have lead to a resurgence in interest for a
solution to manage the proliferation of windows that can happen in any
spatial environment.

The popularity of Tabbed and MDI UIs in some applications is a direct
result of a user's annoyance with the current status of window
management. Tabbing allows the user to not only automatically group
document windows according to application (and thus according to user
meaningful types) -- the GNOME task bar currently has this grouping
feature, but allows the user to apply windowing transformations (resize,
minimize, etc) to all document windows at once. The interaction is not
passive, as a virtual desktop, rather once can actively transform all
windows as a whole group. Currently Tabbed applications allow one to
transform a set of document windows by issuing commands to the parent,
ie "resize GEdit (and by implication all sub-documents currently open
therein)", and has become a crowd favourite. 

The reasons are simple, organization is automatic: 

1. Each sub-document is owned by the parent app, not free-floating or
manually attached to the parent. The parent app represents a semantic
whole, of which the sub-documents are an integral part.

2. The application provides an easy to understand, spatially consistent
UI metaphor for accessing the sub-documents.

3. Any window management action taken on the parent applies uniformly to
the sub-document windows.


1.1.3 Spatial Nautilus

Also, the spatial behaviour of the new Nautilus has highlighted the
existing problems with manually managing UI objects. Each spatial window
of Nautilus adds to the complication of managing that many windows, and
users are rightly concerned. However if we revert or break spatial
Nautilus, the symptoms will be lessened, but the problem will never go
away.

1.2. Existing Solutions on other platforms

Apple has taken a very innovative approach, called expose, with the
technology in OSX which allows the user to apply a couple predefined
transformations to the open windows on a desktop. One thumbnails all
open windows, the second packs all open windows into the screen, the
third packs all open windows belonging to the foreground application
into the screen. While these methods can only categorize windows by
foreground application (3rd method), unlike virtual desktops which can
be arbitrarily categorized, the significant improvement with expose is
that with one keyboard sequence or mouse gesture, one can act upon the
windows in an easily understandable, well-define, spatial way.

1.3. Current Status on Linux/GNOME platform

While XFree lacks the technical ability to graphically mimic Apple's
expose feat, there is nothing stopping the WM from attempting a similar
feature set, although perhaps without the prettiness.

2. Proposal and Implementation

2.1. Solution to Managing Windows

My proposal is two-step, and as far as I can tell, presents no major
technical hurdles: 

First, users need a way to more intuitively group their windows, and
groups of windows, into user-meaningful categories. Human understanding
heavily uses hierarchical categorization, and virtual desktops and task
bars show that we like to organize our windows into groups that hold
meaning to us as individuals. Allowing users to "grab" a set of windows
or set of groups of windows, and form them into a semantic group will
help us do so in a spatially consistent way

Second, users need a way to specify transformations on those groups of
windows. Expose provides such transformations, but they are limited to
three kinds, and have no concept of application groupings, nor any
understanding of the irregular grouping that make sense to us. Using the
arbitrary conceptual groups defined above, and arbitrary transformations
on said groups, we can mimic in desktop space the behaviour our hands,
arms, or other tools might have in real space.

2.2 Solution Implementation Details

The default group is "All Windows or Sub-groups", and can be acted upon
with keyboard shortcuts or mouse gestures when no Sub-groups are in
Focus (ala expose).

2.2.1 Managing Group Lifetimes

The Metacity menu will provide three grouping options, tentatively
called:

1. "Ungroup"

This option changes the mouse icon to a cut lasso (or similar). If the
user then clicks on any grouped window, the component sub-windows or
sub-groups become disassociated and no longer behave as a group.

2. "Group Together"

This option changes the mouse icon to a lasso (or similar), with a user
defined bounding box underneath. Any window or group of windows that
intersects with the bounding box becomes part of the new group.

Metacity then forms a new parent window surrounding the union of all
windows and sub-groups as they preexisted in desktop space. Any
windowing transformation done on the new parent window applies to all
sub-windows and sub-groups. A resize on the parent causes the sub-
windows and sub-groups to resize in proportion.

3. "Stack Together"

This option changes the mouse icon to a lasso (or similar), with a user
defined bounding box underneath. Any window or group of windows that
intersects with the bounding box becomes part of the new group.

Metacity then forms a new parent window, with a horizontal widget-space
below it to hold tabs that represent all the sub-windows and sub-groups
in the new group. Clicking on a tab unmaps the first open tab, and maps
the selected tab.

Also, we want to allow applications to automatically add new documents
to the Metacity Tab bar. This is probably the most technically
challenging since it would requite a change to the WM API, as well as
specs and or protocols. Factoring out the Tabbing behaviour to the
window manager we can reduce application complexity, and retain the
convenience of tabbed organization for applications that benefit from
it.

Tabbed Grouping is only recommended for applications that open a lot of
similar documents, such as a web browser, text browser, or spatial
nautilus. Tabs are conceptually consistent on documents and little else,
but in the case of spatial Nautilus, tabbed elements would look very
much like the path buttons on Seth Nickell's File Chooser UI.

2.2.2 Transforming Groups

1. "Group Together" Mode

	Any translation of the parent window by (x,y) causes the sub-windows
and sub-groups to be recursively translated by (x,y). Any translation of
one of the sub-windows is independent of the other sub-windows. If a
translated sub-window shrinks or grows the original bounding box, then
the parent window shrinks or grows to contain the new position of the
sub-window.

	Any minimize/maximize of the parent window causes the sub-windows and
sub-groups to be recursively minimized/maximized. Any minimize/maximize
of one of the sub-windows causes the window to "roll up"/"unroll"
respectively, since disappearing or exploding windows would break the
conceptual unity of the group.

	Any resize of the parent window by (x%, y%) causes the sub-windows and
sub-groups to be recursively scaled proportional to the parent. This
part will be tricky, keeping the scaling right, and I don't have an
algorithm in mind just yet.

2. "Stack Together" Mode
	
	Any translation of the parent window by (x,y) causes the sub-windows
and sub-groups to be recursively translated by (x,y).

	Any minimize/maximize of the parent window causes the sub-windows and
sub-groups to be recursively minimized/maximized.

	Any resize of the parent window resizes only the currently active tab.
Window sizes are independent of the parent.

3. Interactions and Use Cases

3.1 Spatial Nautilus in Tabbed mode

	The user is searching for a file: User opens the first Nautilus window
en route to the desired file. In the process of "drilling down" into a
directory structure various new spatial windows open, and register as
the current tab in the current parent window. The previous windows exist
and are kept in the Nautilus Group's tabs, but are unmapped so only the
current working folder is visible. Note: this is equivalent to the
manual "Stack" grouping mode.

	Benefits: No explosion of windows. All folders exist, and are spatially
"there", but are grouped in one transformable entity. They are grouped
according to the path taken through the file system, which is a
meaningful categorization, and thus there is a history of all the
folders visited representing a continuous train of exploration. In the
real world Folders are not Documents, although real folders are often
tabbed and arranged in filing cabinets.

	Drawbacks: Fill in here. ;)

3.2 Coding Session with Emacs, gnome-terminal, Mozilla, and Evolution

	The user needs to group 4 distinct applications: After opening all four
apps, the user arranges them about the desktop in the way that is most
comfortable. The user then clicks on a Metacity window's "Group
Together" command, and draws the bounding box around the four apps to
group them. The user can now resize, or move the 4 as if they were one
window.

	Benefits: Although the four apps are unrelated by development, a
programmer will logically think of this set as one unit of windows
needed to perform his task. The window manager bridges this gap by
allowing the user to specify his unique applications sets.

	Drawbacks: Manual creation of the window group.

3.3 Transformations for Managing Ungrouped Windows

	The user needs to find a certain PDF document-window among many open
windows: The default group is the global group of "All windows" so the
user causes, via a certain keyboard shortcut, all windows to tile into
the root window. The user then can select the right PDF window by
reading the unobscured Title bar, and highlighting the choice with the
Tab button. (Behaviour is much like expose is now)

	Benefits: User doesn't have to minimize all open windows, nor route
through a crowded task list for the correct title, but can use his
spatial intuition to "spread out" his open windows across his desk, and
choose the correct one from an overview perspective.

	Drawbacks: XFree currently isn't able to quickly provide visual
feedback about what is in the window, like OSX can.

4. Whats left to discuss?

And here comes the big cop-out you were all waiting for: I like my
ideas, and I think I'm right. But as a student, I don't have time to
implement them until I graduate in the spring. Who agrees with me and
wants to work coding this up?

Also, keep in mind that I'm not some highly trained UI guy.

Cheers,
Ryan