[Mldonkey-bugs] [bugs #1722] file gets downloaded more than 100% - false chunks received

[spiralvoice]

This mail is an automated notification from the bugs tracker
 of the project: mldonkey, a free e-Donkey client.

/**************************************************************************/
[bugs #1722] Latest Modifications:

Changes by: 
                spiralvoice <address@hidden>
'Date: 
                Tue 12/02/2003 at 19:25 (Europe/Berlin)

            What     | Removed                   | Added
---------------------------------------------------------------------------
              Status | Open                      | Closed


------------------ Additional Follow-up Comments ----------------------------
This patch report is very old. If the bug still exists in current

versions please post a new bug report - spiralvoice






/**************************************************************************/
[bugs #1722] Full Item Snapshot:

URL: <http://savannah.nongnu.org/bugs/?func=detailitem&item_id=1722>
Project: mldonkey, a free e-Donkey client
Submitted by: Loial
On: Fri 11/15/2002 at 15:14

Category:  Core
Severity:  5 - Average
Item Group:  None
Resolution:  None
Assigned to:  None
Status:  Closed
Release:  2.00
Release:  
Platform Version:  Linux i386-i686
Binaries Origin:  Compiled From CVS


Summary:  file gets downloaded more than 100% - false chunks received

Original Submission:  I'm having difficulty downloading some files correctly 
because I download more than 100% of the file.

This is what I think happens:

I download a file, 
e.g.ed2k://|file|Need_For_Speed_Hot_Pursuit_2.DEVIANCE.ShareReactor.bin|739678128|CEDEDC2F223B6FC7A9B0AC651A686FBD|/



but someone seems to have some incorrect chunks, because the downloaded size is 
bigger than the original file size. Pressing 'Verify chunks' removes the 
incorrect data en leaves me with a file that is e.g. 95% downloaded. This is 
good. However the incorrect chunks are downloaded again, so I again have a file 
that is not correct and I have to press 'Verify chunks' again. This keeps 
repeating itself.

The problem is, I think, that the incorrect chunk is downloaded from the same 
user(s) again. It would be better though if the username is remembered as 
having that incorrect part, so the part is downloaded from someone else.

Problem might be that during transmission of the original correct chunk the 
data got corrupted, resulting in the now incorrect chunk and then it would not 
be wise to remember that user as having an incorrect chunk. Or is this no 
problem, because it is already taken care of by the transmission protocol?

It can be said that there isn't a real problem here, because eventually the 
missing part will be downloaded from someone else and then the chunk will be 
correct and the file will be finished. However at the moment I am downloading 
at 100kb/s for half a day already and I still only have the same 90% of the 
file that I had this morning. This is ofcourse a big waste of network bandwith 
and should therefor be prevented.

I hope I could make myself clear :)

Goodluck and thanx

Follow-up Comments
------------------


-------------------------------------------------------
Date: Tue 12/02/2003 at 19:25       By: spiralvoice
This patch report is very old. If the bug still exists in current

versions please post a new bug report - spiralvoice

-------------------------------------------------------
Date: Tue 08/26/2003 at 10:53       By: None
Last time I looked at the officiel eMule sources, ICH implementation was 
actually much more simple than that. When corruption is found, chunk is 
invalidated (but data not cleared).

Download starts over, overwriting previous data as it arrives; But md4 hash is 
recomputed each time, so eventually the chunk may get validated before being 
fully redownloaded...



-------------------------------------------------------
Date: Fri 11/22/2002 at 01:30       By: None
I could not find a specification of the ICH (Intelligent Corruption Handling) 
protocol extension, but it looks rather simple.

With standard edonkey protocol, you fetch chunks (9.5Mb) md4 hashes before 
downloading, then each time you complete a chunk, you use its reference hash to 
validate the chunk. If it doesn't match, the whole chunk is invalidated.

With ICH, when corruption is detected, you ask for md4 hashes of zones (180Kb) 
inside the chunk to another peer, and individually validate or invalidate 
zones, then only refetch corrupted zones...



-------------------------------------------------------
Date: Tue 11/19/2002 at 16:05       By: Loial
Yesterday the download finished. It took me quite a while to get the last few 
percent. I have no idea how much false chunks I've downloaded, although I have 
seen the downloaded percentage wrap around and display a negative number once 
or twice ;-)



I do not know what you mean with 'the eMule's ICH Protocol extension', so I 
can't comment on that.

-------------------------------------------------------
Date: Sun 11/17/2002 at 16:17       By: None
I tried downloading this file (giving the whole ed2k URL wasn't very wise; next 
time, if you really need to, only give MD4 hash and size, it's enough)

Download went to 101.6% or so, and the file completed normally, without my 
intervention...



-------------------------------------------------------
Date: Sat 11/16/2002 at 21:11       By: None
IP packets are checksummed, so corruption during transfer should be detected; 
But they're other sources of corruption (disk, memory,... of a peer)

The problem is that a given chunk doesn't usually come from just one source, 
but from several sources, so mldonkey would have to keep a list of involved 
sources for each chunk...

Another solution would be to implement the eMule's ICH protocol extension (or a 
similar extension).












For detailed info, follow this link:
<http://savannah.nongnu.org/bugs/?func=detailitem&item_id=1722>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.nongnu.org/