Re: [gnome-network]downman description

[Rodrigo Moya <rodrigo gnome-db org>]

On Mon, 2003-09-22 at 14:59, Manuel Clos wrote:
> Hi all,
> 
> I'll try to do some downman marketing :)
> 
afer looking a bit at the code, I've got more comments, and more worries
about downman.

> downman is modeled after the idea of having a separate proces, a daemon, 
> that takes care of downloading / uploading files. I found this idea 
> first in darxite and then implemented it in downman. It is much like 
> Rodney said: "... I have been planning on rewriting it from scratch to 
> use a separate daemon backend and gui frontend code ..." when refering 
> to emphetamine.
> 
yes, that's the general idea, but after looking at the daemon code in
downman, I just wonder what we can reuse there, if anything at all. It
doesnt use gnome-vfs, it relies on opening FTP/HTTP sockets. It should
use the async interface in gnome-vfs instead of using threads, and I
cant see the way your current code can be changed to that without being
re-implemented.

> The daemon is per user, like gcond. It must be started when the first 
> application needs it (nowadays Gnome Download Manager).
> 
hmm, the second thing I dont like about downman again is how the daemon
is implemented. It seems from the code that it uses UNIX sockets to do
the client->daemon communication. Am I right? If so, that code, again,
serves no purpose at all for the Trasnfer Manager in GO, since we
already have Bonobo to do that communication. So all that code from
downman has to be rewritten also.

> The daemon consists on a simple thread where a GMainLoop waits for 
> events. These events can be clients trying to connect, clients sending 
> commands or downloads changing state.
>
string commands right? That's what I think I've understood from the
code.

>  Like many other download managers, 
> downman can manage different lists of downloads. This lists are called 
> Projects. When you first start downman you only have one project. In 
> Gnome Download Manager you can see this because it will show you no 
> tabs, this is, it will look like a simple list.
>
this sounds like a good idea

>  The Project GObject 
> takes care of the downloads. Every seconds it runs the scheduler to see 
> if there are any queued download and starts it if possible. Here is also 
> where the speed limits are calculated.
> 
> Then, there is the Download class. Every download has some properties 
> like url, destination dir, speed limit, current downloaded bytes, status 
> of the download, ... Every download runs on its own thread. When running 
> it goes in a loop to read and write data and checking for events until 
> it finishes.
> 
Download class? where is that class? You mean the one in daemon?

> When a download finishes, changes its status or something happens, a 
> g_signal is emitted so that the Project knows what has happened. The 
> project then emits a signal so that Downman (the daemon class) can 
> notify the clients that have requested to be sent events.
> 
how are clients notified? via the UNIX sockets stuff?

Again, all this is not needed at all in the gnome-vfs case, since
gnome-vfs does evertything for us (except notifying the clients, which
we can do with a BonoboListener, for instance). So, again, there is not
much code we can reuse here :-(

> At this point you have the basic functionality
> 
> clients <------------------------> daemon <------------------------> 
> projects <------------------> downloads / uploads
> 
> To make it easier, there is a util library that implements the commands 
> as a C API, so you have:
> 
> gint        dm_client_local_connect (gchar *name);
>
what does local_connect mean? That you can connect to remote download
managers also (ie, is there a remote_connect call)?

> gboolean    dm_client_daemon_start (GError **error);
> 
this is not needed at all in the Bonobo-based case, since the daemon
will be started when activating it, it it's not been started before.

> which allow clients to connect and to start the daemon.
> 
> GString *   dm_command_get_new          (guint project, const gchar *url);
> gint    dm_command_get                  (GString *item, guint fd);
> 
what is the 'fd' stuff, that is also in client_new in daemon/? Is that
the UNIX socket (or whatever) you use to communicate with the daemon?

> this is the minimun command, you can also set other properties like 
> localpath, speed, ...
> 

> The utility lib also has the common commands:
> 
> gint    dm_command_download_start   (gint fd, guint project, guint 
> download);
> gint    dm_command_download_stop    (gint fd, guint project, guint 
> download);
> gint    dm_command_download_queue   (gint fd, guint project, guint 
> download);
> gint    dm_command_download_delete  (gint fd, guint project, guint 
> download);
> 
again, what is the purpose of exposing a 'fd' in the API?

> 
> At this point, creating clients is just a matter of calling the 
> local_connect function and adding a download.
> 
> Clients in downman are classified under the clients tree, where you can 
> have CLI clients (downman) or GUI clients (Gnome Download Manager)
> 
> Gnome Download Manager uses all the features that are implemented. It 
> connects to the daemon and shows the list of downloads, where you will 
> see no tabs when there is only a project, and a GtkNotebook when there 
> are more than one project. You will have the list of downloads along 
> with the status, % done, ....
>
this is nice, yes, but belongs to the UI, which I guess is the second
part, after we have done the basic framework.

>  The GUI does not block because it uses 
> events in the file descriptor to read the info coming from the daemon. 
> It allows you to add downloads, start, stop, delete, ... and set the 
> properties of the donwloads (even speedlimit in real time). I have 
> always followed the HIG when doing the GUI. It uses glade for easy 
> modification.
> 
> So as Gnome Download Manager only uses the util lib, moving the daemon 
> and util lib to gnome-network will mean that it only needs to link 
> against libgnomenetwork. Of course, moving the GUI will be great too.
> 
moving the daemon and util lib is useless since, as I've understood, we
have to rewrite a lot of things, as stated above.

> There is also a floating window (downman-gmonitor) where you can drop urls.
> 
that might be a good idea also, although I guess it is much better to
have the daemon display an icon in the system tray, and make it accept
dropped URIs.

> As every download manager, downman saves the list of downloads, it does 
> so in using libxml. And the same is true for downman and projects 
> configuration.
> 
> There is now a Servers feature, where all servers that have a maximum 
> number of connections get saved to the servers.xml file. Of course, this 
> can be expanded to include more server properties.
> 
> I have been always writing downman for integration with gnome2, so I 
> used glib as the base for the daemon and utillib.
>
well, to integrate with GNOME you have to use more than glib :-) First
of all, you dont have to reimplement things that are already available,
like the sockets/whatever communication you have in the client->daemon
case.

>  Followed the HIG when 
> creating GUIs. Using libxml for the files. GIOChannel for reading and 
> writing. GThread. GString and so on.
> 
following the HIG is the only thing of these that makes it to integrate
with GNOME :-) Using data types in glib is nice, but not enough.

I have to say though that I generally like the downman UI.

> It much provides what rodrigo wants:
> 	* basic functionality in libgnetwork   --> this is util lib + the daemon
> 	* try to push that library to be used in other apps   --> a better util 
> lib API and/or a bonobo Interface
> 	* have a nice GUI   --> Gnome Download Manager
> 
> Plus other itheas like:
> 	- daemon / GUI separation
> 
right now, if I understand well, you dont have this separation, so
again, a lot of work (maybe a full reimplementation) of the daemon seems
to be needed.

> Next in my TODO list is:
> 	- Moving to use gnome-vfs everywhere
>
this will mean rewriting most things in the daemon, which is similar
than starting from scratch, which is what Rodney is planning.

> 	- Renaming / restructuration of the code if it is being moved to 
> gnome-network
> 
> I really want to help gnome-network to get a good _transfer_ manager. 
> And I think downman is a good starting point. I invite to Tapia and 
> Rodney to take a look at downman and help with the move to gnome-network.
> 
> Downman has been working for some time and is pretty stable.
> 
yes, but the architecture does not fit, since there are a lot of things
that need to be reimplemented.

> I have also looked at Tapia code, but it is pretty simple and puts 
> together the download and daemon information. There is need for a 
> separate download / upload / transfer class and leave the daemon to do 
> only the scheduling, client management, saving configuration and list of 
> files, much like downman does now.
> 
> So now I'm waiting on people telling what are the problems for going in. 
> As I said before, if downman gets in now, we can have a pretty good and 
> rock stable transfer manager for 2.6, since almost all work is already done.
>
Dude, it's not done :-) If I've understood correctly (and sorry if I
didn't), most things need to be reimplemented to fit gnome-vfs and
Bonobo in. The client->daemon stuff is to be rewritten, using a CORBA
interface, Bonobo listeners to notify clients, etc. The daemon code
needs to be reimplemented to remove all the FTP/HTTP protocol stuff and
replace it with gnome-vfs, which already has the asynchonous stuff.
Also, it seems the daemon code only supports downloads, since it does
not seem to contain code in the HTTP/FTP code to log in and upload a
file. Or maybe I missed it.

Sorry if I sound a bit unpolite, it is not my intention at all, but you
keep saying that downman fits in with the archictecture we're talking
about, and from what I've seen, it needs almost to be
rewritten/re-architected to match that plan. I'd like to be wrong, and
really have downman be the solution to this discussion. This would end
up my current headaches of tying to keep you guys in agreement :-) and
will make you work in other gnome-network things.

I must say, to not sound so harsh to you, that the UI of gdownman is
nice, and if nobody has a better UI idea, we might end up reusing a lot
of your UI.

cheers