gvfsd-http runs out of file descriptors: many sockets in CLOSE_WAIT state

[Matteo Settenvini <matteo-ml member fsf org>]

Dear gvfs developers,

it's some times I have noticed an unwanted behaviour of gvfsd-http.
That is, sockets used to retrieve files over the network stay in the
CLOSE_WAIT state for ages... and thus the number of FDs is growing and
growing. I have a long-running program I wrote (and I know also for
Rhythmbox this was an issue), which soon runs out of file descriptors,
leading to a failure.

To help better seeing the problem, I have written a small program using
Gtkmm, which exhibits the issue. It is attached here. To see it run out
of file descriptors, increase the amount of fetches to 1001.

I tried digging into GDaemonFile's source, which I think is the client
counterpart to gvfsd-http, and I *think* the problem is at least that,
upon destruction of the GFile it subtypes, gvfsd-http does not get
notified to close the socket. In alternative, closing the socket upon a
call to g_file_read_finish(), inside GIO, might be the correct think to
do.

I would like to know if this is a bug in gvfs, gio, or my program, and
in the former cases if I should submit it to Bugzilla.

Thanks for your help!
Matteo

% ---------------- %< ----------------------- %< ---------------------

// Please compile with:
//      g++ -std=c++11 `pkg-config --cflags --libs glibmm-2.4 giomm-2.4`test.cc
// Then kill gvfsd-http before running, and monitor with:
//      lsof -p $(pidof gvfsd-http)

#include <glibmm.h>
#include <giomm.h>

#include <iostream>

namespace {
        static const char *URL = "http://www.google.com/images/srpr/logo11w.png";;
}

using Glib::ustring;
using Glib::RefPtr;

static RefPtr<Glib::MainLoop> main_loop = Glib::MainLoop::create ();

class RemoteLoader
{
public:
        RemoteLoader (const Glib::ustring& url)
          : m_cancellable (Gio::Cancellable::create ())
        {
                std::cout << "Requesting " << url << std::endl;
                auto file = Gio::File::create_for_uri (url);
                auto callback = sigc::ptr_fun (&RemoteLoader::on_load);
                file->load_contents_async (callback, m_cancellable);
        }

        ~RemoteLoader ()
        {
                auto cancellable = m_cancellable;
                if (!cancellable) return;

                Glib::Thread *joinable = Glib::Thread::create ([cancellable] ()
                {       
                        cancellable->cancel ();
                        // No direct support for detached threads, so...
                        Glib::Thread *self = Glib::Thread::self ();
                        g_thread_unref (self->gobj ());
                });
        }

        static void on_load (const RefPtr<Gio::AsyncResult>& result)
        {
                char *buf = nullptr;
                gsize length = 0;

                const auto file = RefPtr<Gio::File>::cast_dynamic (result->get_source_object_base ());
                if (!file) return;

                try
                {
                        bool success = file->load_contents_finish (result, buf, length);
                        if (!success) 
                        {
                                std::cerr << "Not successful (" << file->get_uri () << ")" << std::endl;
                        }
                        else
                        {
                                std::cout << "Successful load (" << file->get_uri () << ")" << std::endl;
                        }
                }
                catch (const Glib::Error& e)
                {
                        std::cerr << "Exception: " << e.what () << " when trying to load "
                                  << file->get_uri () << std::endl;
                }

                g_free (buf);
        }


private:
        RefPtr<Gio::Cancellable> m_cancellable;
};


static bool
load_another ()
{
        static int how_many = 0;
        if (how_many > 9)
        {
                std::cout << "Exiting in 5 seconds... " << std::endl;
                Glib::signal_timeout ().connect_seconds_once ([] { main_loop->quit (); }, 5);
                return false;
        }
        
        for (int i = how_many*5; i < (how_many+1)*5; ++i)
        {
                std::ostringstream oss;
                oss << URL << "?nocache=" << i;
                auto loader = new RemoteLoader (oss.str ());
                Glib::signal_timeout ().connect_seconds_once ([loader] { delete loader; }, 5);
        }

        ++how_many;
        return true;
}


int
main ()
{
        std::cout << " === Killing gvfsd-http === " << std::endl;
        system ("killall gvfsd-http");

        Gio::init ();
        Glib::signal_timeout ().connect_seconds (sigc::ptr_fun (&load_another), 1);
        main_loop->run ();

        std::string cmd = "bash -c 'pidof gvfsd-http | xargs lsof -p'";
        std::cout << "\n\n\t === Running " << cmd << " === \n" << std::endl;
        return system (cmd.c_str ());
}

Attachment: pgpr_aPIc22iE.pgp
Description: PGP signature