Discussion of libtorrent, a C++ bittorrent library.

Arvid Norberg | 5 Jan 2007 02:34

Re: filter_files() implementation

Hi.

I have been thinking about this problem a bit on and off. It is a  
feature that is quite requested.
So far I haven't found a solution that I'm satisfied with (either  
because of lack of functionality or complexity in implementation). I  
wrote a post about this some time ago, it can be found here:

http://article.gmane.org/gmane.network.bit-torrent.libtorrent/829

So, in short, the "problem" is that the storage need a more advanced  
interface where it is possible to tell it which files should not be  
created, and it should instead map some data to some file only  
containing left overs. This also requires more logic in the initial  
checker, since it need to take into account that some parts of some  
pieces may be stored at other places.

On Jan 4, 2007, at 5:45 , James Longstreet wrote:

> Hello,
>
> The main selling point of the client I'm working on is selective
> downloading.  It just about couldn't be easier than
> torrent_handle::filter_files().
>
> The only problem is the pieces that overlap between a filtered file  
> and
> an unfiltered file.  Obviously we have to download these pieces,  
> and we
> have to write at least the parts of the pieces corresponding to

(Continue reading)

headers

James Longstreet | 5 Jan 2007 19:47

Re: filter_files() implementation

I've been looking over Azureus' implementation, and it seems like they 
use a cache file of sorts.  Their code is quite messy and I'm not a Java
expert, so I'm still trying to dig through it.

First, they have the concept of filtering files, but not pieces.  For 
each piece they keep a list of file entries, containing the file index,
offset, and length of that file's portion of the piece.  So when the 
piece picker considers a piece, it checks each file that that piece 
spans, and if all of them are filtered, it doesn't download the piece.
I haven't yet figured out how they store the cache file or map the
pieces for checking.

It seems to me that the way to do it is to store the entire piece in the
cache file, instead of just the portion that doesn't map to a selected
file.

Another problem is where on the filesystem the cache is stored.  It
would need to be done in a platform-independent way, and temporary files
would of course be ideal, but at the same time, it should be persistent,
which suggests not using temporary files, and it might be rather large,
which suggests not using /tmp or equivalent.

I'll look into possible implementations of this more, and see what I
come up with.  I'm willing to make the changes (or at least help),
because I'd really like to see this.

James Longstreet

On Fri, Jan 05, 2007 at 02:34:42AM +0100, Arvid Norberg wrote:
> Hi.

(Continue reading)