Merged bug fixes from trunk.

2008-01-10  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Fixed the bug that EX_TOO_LONG_PAYLOAD exception is thrown if 
just
	payload length(4bytes) are received. This happens because 
lenbufLength
	is not updated in this particular case and successive call of
	receiveMessage() overwrites payload length with bytes recieved 
which
	are payload body.
	* src/PeerConnection.{h, cc}
	* src/message.h

2008-01-09  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Removed a call to isPowerOf() because it is no longer necessary 
here
	and a request block is not always power of 2.
	BUG#1866924
	* src/PeerMessageUtil.cc (checkLength)

2008-01-07  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Fixed the bug that always first found Segment is removed from
	usedSegmentEntries. Removed unused functions.
	* src/SegmentMan.{h, cc}
	* test/SegmentManTest.cc

2008-01-07  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Fixed the bug that SegmentMan::completeSegment() is not called
	even if Segment is complete when --lowest-speed-limit is 
enabled.
	BUG#1864525
	* src/DownloadCommand.{h, cc}

2008-01-06  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Fixed: hash algorithm 'sha1' is always used. 
	* src/DownloadCommand.cc (validatePieceHash) 

2007-12-26  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Fixed memory leak
	* src/Piece.cc
	
2007-12-22  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Fixed the bug that causes aria2 not to finish download. 
BUG#1855875.
	I could reproduce this bug in following procedure:
	1. Stop the download at the very beginning(1% or 100KB 
downloaded).
	2. Restart aria2.
	3. You see the download stopped around 99%.
	* src/HttpResponseCommand.cc (handleDefaultEncoding)
	* src/StreamFileAllocationEntry.cc: Removed the timeout 
handling.
	If timeout is reached, then _nextCommand is unused and it may 
contains
	segments and they won't be canceled. Actually, timeout is not 
needed
	here because if the server dropped connection, then retry is 
made.

2007-12-22  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Fixed the bug that returns incomplete data when it contains 
null
	character. A convenient constructor was also added.
	* src/Data.{h, cc}

2007-12-16  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
	
	Added "Status Legend" label to the explanation text of 'stat' 
in
	download result and moved it to the last. BUG#1848214
	* src/RequestGroupMan.cc

2007-12-15  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Fixed the bug that prevents aria2 from loading cookie file when 
expire
	value is greater than 2^31-1. BUG#1851066
	* src/CookieBoxFactory.cc
	* test/CookieBoxFactoryTest.cc

2007-12-14  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Fixed possible memory leak when an exception is thrown.
	* src/XML2SAXMetalinkProcessor.cc

2007-12-10  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Compiler error fix: applied the patch by Tiziano Mueller
	* src/RequestGroup.cc

2007-12-10  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>

	Fixed the bug: only first announce URL is tried in 
AnnounceTier,
	in stopped and completed event.
	* src/AnnounceList.{h, cc}
	* test/AnnounceListTest.cc
	* src/DefaultBtAnnounce.cc
	* test/DefaultBtAnnounceTest.cc

	Sorted URLs.
	* test/Metalink2RequestGroupTest.cc

ChangeLog

@@ -1,3 +1,98 @@
+2008-01-10  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Fixed the bug that EX_TOO_LONG_PAYLOAD exception is thrown if just
+	payload length(4bytes) are received. This happens because lenbufLength
+	is not updated in this particular case and successive call of
+	receiveMessage() overwrites payload length with bytes recieved which
+	are payload body.
+	* src/PeerConnection.{h, cc}
+	* src/message.h
+
+2008-01-09  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Removed a call to isPowerOf() because it is no longer necessary here
+	and a request block is not always power of 2.
+	BUG#1866924
+	* src/PeerMessageUtil.cc (checkLength)
+
+2008-01-07  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Fixed the bug that always first found Segment is removed from
+	usedSegmentEntries. Removed unused functions.
+	* src/SegmentMan.{h, cc}
+	* test/SegmentManTest.cc
+
+2008-01-07  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Fixed the bug that SegmentMan::completeSegment() is not called
+	even if Segment is complete when --lowest-speed-limit is enabled.
+	BUG#1864525
+	* src/DownloadCommand.{h, cc}
+
+2008-01-06  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Fixed: hash algorithm 'sha1' is always used. 
+	* src/DownloadCommand.cc (validatePieceHash) 
+
+2007-12-26  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Fixed memory leak
+	* src/Piece.cc
+	
+2007-12-22  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Fixed the bug that causes aria2 not to finish download. BUG#1855875.
+	I could reproduce this bug in following procedure:
+	1. Stop the download at the very beginning(1% or 100KB downloaded).
+	2. Restart aria2.
+	3. You see the download stopped around 99%.
+	* src/HttpResponseCommand.cc (handleDefaultEncoding)
+	* src/StreamFileAllocationEntry.cc: Removed the timeout handling.
+	If timeout is reached, then _nextCommand is unused and it may contains
+	segments and they won't be canceled. Actually, timeout is not needed
+	here because if the server dropped connection, then retry is made.
+
+2007-12-22  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Fixed the bug that returns incomplete data when it contains null
+	character. A convenient constructor was also added.
+	* src/Data.{h, cc}
+
+2007-12-16  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+	
+	Added "Status Legend" label to the explanation text of 'stat' in
+	download result and moved it to the last. BUG#1848214
+	* src/RequestGroupMan.cc
+
+2007-12-15  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Fixed the bug that prevents aria2 from loading cookie file when expire
+	value is greater than 2^31-1. BUG#1851066
+	* src/CookieBoxFactory.cc
+	* test/CookieBoxFactoryTest.cc
+
+2007-12-14  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Fixed possible memory leak when an exception is thrown.
+	* src/XML2SAXMetalinkProcessor.cc
+
+2007-12-10  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Compiler error fix: applied the patch by Tiziano Mueller
+	* src/RequestGroup.cc
+
+2007-12-10  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
+
+	Fixed the bug: only first announce URL is tried in AnnounceTier,
+	in stopped and completed event.
+	* src/AnnounceList.{h, cc}
+	* test/AnnounceListTest.cc
+	* src/DefaultBtAnnounce.cc
+	* test/DefaultBtAnnounceTest.cc
+
+	Sorted URLs.
+	* test/Metalink2RequestGroupTest.cc
+
 2007-12-09  Tatsuhiro Tsujikawa  <tujikawa at rednoah dot com>
 
 	Removed unnecessary string copy. Updated doc and corrected indentation.

doc/aria2c.1

@@ -1,11 +1,11 @@
 .\"     Title: aria2c
 .\"    Author: 
 .\" Generator: DocBook XSL Stylesheets v1.73.1 <http://docbook.sf.net/>
-.\"      Date: 12/08/2007
+.\"      Date: 12/10/2007
 .\"    Manual: 
 .\"    Source: 
 .\"
-.TH "ARIA2C" "1" "12/08/2007" "" ""
+.TH "ARIA2C" "1" "12/10/2007" "" ""
 .\" disable hyphenation
 .nh
 .\" disable justification (adjust text to left margin only)
@@ -20,7 +20,7 @@ aria2c - The ultra fast download utility
 \fIaria2c\fR [OPTIONS] \-M METALINK_FILE
 .sp
 .SH "DESCRIPTION"
-aria2 is a utility for downloading files\. The supported protocols are http(s)/ftp/BitTorrent/Metalink\. aria2 has powerful segmented downloading ability, downloading a file from multiple sources and multiple protocols, utilizing your download bandwidth to the max\. It even supports downloading a file from http(s)/ftp and BitTorrent at the same time, while the data downloaded form http(s)/ftp is uploaded to BitTorrent swarm\.
+aria2 is a utility for downloading files\. The supported protocols are http(s)/ftp/BitTorrent/Metalink\. aria2 has powerful segmented downloading ability, downloading a file from multiple sources and multiple protocols, utilizing your download bandwidth to the max\. It even supports downloading a file from http(s)/ftp and BitTorrent at the same time, while the data downloaded from http(s)/ftp is uploaded to BitTorrent swarm\.
 .sp
 aria2 also provides most reliable http(s)/ftp downloading experience ever\. Using Metalink\'s chunk checksums, aria2 automatically validates chunk of data while downloading a file like BitTorrent\.
 .sp

doc/aria2c.1.txt

@@ -20,7 +20,7 @@ aria2 is a utility for downloading files.
 The supported protocols are http(s)/ftp/BitTorrent/Metalink.
 aria2 has powerful segmented downloading ability, downloading a file
 from multiple sources and multiple protocols, utilizing your download bandwidth to the max.
-It even supports downloading a file from http(s)/ftp and BitTorrent at the same time, while the data downloaded form http(s)/ftp is uploaded to BitTorrent swarm.
+It even supports downloading a file from http(s)/ftp and BitTorrent at the same time, while the data downloaded from http(s)/ftp is uploaded to BitTorrent swarm.
 
 aria2 also provides most reliable http(s)/ftp downloading experience ever.
 Using Metalink's chunk checksums, aria2 automatically validates chunk of data

src/AnnounceList.cc

@@ -237,3 +237,21 @@ void AnnounceList::resetTier()
 {
   resetIterator();
 }
+
+bool AnnounceList::currentTierAcceptsStoppedEvent() const
+{
+  if(currentTrackerInitialized) {
+    return FindStoppedAllowedTier()(*currentTier);
+  } else {
+    return false;
+  }
+}
+
+bool AnnounceList::currentTierAcceptsCompletedEvent() const
+{
+  if(currentTrackerInitialized) {
+    return FindCompletedAllowedTier()(*currentTier);
+  } else {
+    return false;
+  }
+}

src/AnnounceList.h

@@ -119,6 +119,10 @@ public:
   bool allTiersFailed() const;
 
   void resetTier();
+
+  bool currentTierAcceptsStoppedEvent() const;
+
+  bool currentTierAcceptsCompletedEvent() const;
 };
 
 #endif // _D_ANNOUNCE_LIST_H_

src/CookieBoxFactory.cc

@@ -35,6 +35,7 @@
 #include "CookieBoxFactory.h"
 #include "CookieParser.h"
 #include "Util.h"
+#include "RecoverableException.h"
 
 CookieBoxHandle CookieBoxFactory::createNewInstance()
 {
@@ -50,9 +51,15 @@ void CookieBoxFactory::loadDefaultCookie(istream& s)
     if(Util::startsWith(line, "#")) {
       continue;
     }
-    Cookie c = parseNsCookie(line);
-    if(c.good()) {
-      defaultCookies.push_back(c);
+    try {
+      Cookie c = parseNsCookie(line);
+      if(c.good()) {
+	defaultCookies.push_back(c);
+      }
+    } catch(RecoverableException* e) {
+      // ignore malformed cookie entry
+      // TODO better to log it
+      delete e;
     }
   }
 }
@@ -68,7 +75,12 @@ Cookie CookieBoxFactory::parseNsCookie(const string& nsCookieStr) const
   c.domain = vs[0];
   c.path = vs[2];
   c.secure = vs[3] == "TRUE" ? true : false;
-  c.expires = Util::parseInt(vs[4]);
+  int64_t expireDate = Util::parseLLInt(vs[4]);
+  // TODO assuming time_t is int32_t...
+  if(expireDate > INT32_MAX) {
+    expireDate = INT32_MAX;
+  }
+  c.expires = expireDate;
   c.name = vs[5];
   if(vs.size() >= 7) {
     c.value = vs[6];

src/Data.cc

@@ -46,21 +46,24 @@ Data::Data(const char* data, int32_t len, bool number):number(number) {
   }
 }
 
+Data::Data(const string& data, bool number):number(number)
+{
+  if(data.empty()) {
+    this->data = 0;
+    this->len = 0;
+  } else {
+    this->data = new char[data.size()];
+    memcpy(this->data, data.c_str(), data.size());
+    this->len = data.size();
+  }
+}
+
 Data::~Data() {
   delete [] data;
 }
 
 string Data::toString() const {
-  if(len == 0) {
-    return "";
-  } else {
-    char* temp = new char[len+1];
-    memcpy(temp, data, len);
-    temp[len] = '\0';
-    string str(temp);
-    delete [] temp;
-    return str;
-  }
+  return string(&data[0], &data[len]);
 }
 
 const char* Data::getData() const {

src/Data.h

@@ -51,6 +51,9 @@ public:
    * memory of data.
    */
   Data(const char* data, int32_t len, bool number = false);
+
+  Data(const string& data, bool number = false);
+
   ~Data();
 
   string toString() const;

src/DefaultBtAnnounce.cc

@@ -100,10 +100,14 @@ bool DefaultBtAnnounce::isAnnounceReady() {
 
 string DefaultBtAnnounce::getAnnounceUrl() {
   if(isStoppedAnnounceReady()) {
-    announceList.moveToStoppedAllowedTier();
+    if(!announceList.currentTierAcceptsStoppedEvent()) {
+      announceList.moveToStoppedAllowedTier();
+    }
     announceList.setEvent(AnnounceTier::STOPPED);
   } else if(isCompletedAnnounceReady()) {
-    announceList.moveToCompletedAllowedTier();
+    if(!announceList.currentTierAcceptsCompletedEvent()) {
+      announceList.moveToCompletedAllowedTier();
+    }
     announceList.setEvent(AnnounceTier::COMPLETED);
   } else if(isDefaultAnnounceReady()) {
     // If download completed before "started" event is sent to a tracker,

src/DownloadCommand.cc

@@ -118,16 +118,6 @@ bool DownloadCommand::executeInternal() {
     //segment->writtenLength += infbufSize;
     peerStat->updateDownloadLength(infbufSize);
   }
-  // calculate downloading speed
-  if(peerStat->getDownloadStartTime().elapsed(startupIdleTime)) {
-    int32_t nowSpeed = peerStat->calculateDownloadSpeed();
-    if(lowestDownloadSpeedLimit > 0 &&  nowSpeed <= lowestDownloadSpeedLimit) {
-      throw new DlAbortEx(EX_TOO_SLOW_DOWNLOAD_SPEED,
-			  nowSpeed,
-			  lowestDownloadSpeedLimit,
-			  req->getHost().c_str());
-    }
-  }
   if(_requestGroup->getTotalLength() != 0 && bufSize == 0) {
     throw new DlRetryEx(EX_GOT_EOF);
   }
@@ -137,14 +127,30 @@ bool DownloadCommand::executeInternal() {
     if(!transferDecoder.isNull()) transferDecoder->end();
     logger->info(MSG_SEGMENT_DOWNLOAD_COMPLETED, cuid);
     validatePieceHash(segment);
+    checkLowestDownloadSpeed();
     // this unit is going to download another segment.
     return prepareForNextSegment();
   } else {
+    checkLowestDownloadSpeed();
     e->commands.push_back(this);
     return false;
   }
 }
 
+void DownloadCommand::checkLowestDownloadSpeed() const
+{
+  // calculate downloading speed
+  if(peerStat->getDownloadStartTime().elapsed(startupIdleTime)) {
+    int32_t nowSpeed = peerStat->calculateDownloadSpeed();
+    if(lowestDownloadSpeedLimit > 0 &&  nowSpeed <= lowestDownloadSpeedLimit) {
+      throw new DlAbortEx(EX_TOO_SLOW_DOWNLOAD_SPEED,
+			  nowSpeed,
+			  lowestDownloadSpeedLimit,
+			  req->getHost().c_str());
+    }
+  }
+}
+
 bool DownloadCommand::prepareForNextSegment() {
   if(_requestGroup->downloadFinished()) {
 #ifdef ENABLE_MESSAGE_DIGEST
@@ -179,7 +185,7 @@ void DownloadCommand::validatePieceHash(const SegmentHandle& segment)
   if(e->option->get(PREF_REALTIME_CHUNK_CHECKSUM) == V_TRUE &&
      !expectedPieceHash.empty()) {
     string actualPieceHash =
-      MessageDigestHelper::digest("sha1",
+      MessageDigestHelper::digest(_requestGroup->getDownloadContext()->getPieceHashAlgo(),
 				  _requestGroup->getPieceStorage()->getDiskAdaptor(),
 				  segment->getPosition(),
 				  segment->getLength());

src/DownloadCommand.h

@@ -51,6 +51,7 @@ private:
 
   void validatePieceHash(const SegmentHandle& segment);
 
+  void checkLowestDownloadSpeed() const;
 protected:
   TransferEncodingHandle transferDecoder;
 

src/HttpResponseCommand.cc

@@ -135,6 +135,8 @@ bool HttpResponseCommand::handleDefaultEncoding(const HttpResponseHandle& httpRe
     SegmentHandle segment = _requestGroup->getSegmentMan()->getSegment(cuid, 0);
     if(!segment.isNull() && segment->getPositionToWrite() == 0) {
       command = createHttpDownloadCommand(httpResponse);
+    } else {
+      _requestGroup->getSegmentMan()->cancelSegment(cuid);
     }
     prepareForNextAction(command);
     e->noWait = true;

src/PeerConnection.cc

@@ -71,29 +71,25 @@ int32_t PeerConnection::sendMessage(const unsigned char* data, int32_t dataLengt
 }
 
 bool PeerConnection::receiveMessage(unsigned char* data, int32_t& dataLength) {
-  if(!socket->isReadable(0)) {
-    return false;
-  }
-  if(resbufLength == 0 && lenbufLength != 4) {
-    // read payload size, 4-byte integer
-    int32_t remain = 4-lenbufLength;
-    int32_t temp = remain;
-    // TODO fix this
-    socket->readData((char*)lenbuf+lenbufLength, temp);
-    if(temp == 0) {
+  if(resbufLength == 0 && 4 > lenbufLength) {
+    if(!socket->isReadable(0)) {
+      return false;
+    }
+    // read payload size, 32bit unsigned integer
+    int32_t remaining = 4-lenbufLength;
+    socket->readData(lenbuf+lenbufLength, remaining);
+    if(remaining == 0) {
       // we got EOF
       throw new DlAbortEx(EX_EOF_FROM_PEER);
     }
-    if(remain != temp) {
-      // still 4-temp bytes to go
-      lenbufLength += temp;
+    lenbufLength += remaining;
+    if(4 > lenbufLength) {
+      // still 4-lenbufLength bytes to go
       return false;
     }
-    //payloadLen = ntohl(nPayloadLen);
-    int32_t payloadLength = ntohl(*((int32_t*)lenbuf));
-    if(payloadLength > MAX_PAYLOAD_LEN || payloadLength < 0) {
-      throw new DlAbortEx(EX_TOO_LONG_PAYLOAD,
-			  payloadLength);
+    uint32_t payloadLength = ntohl(*(reinterpret_cast<uint32_t*>(lenbuf)));
+    if(payloadLength > MAX_PAYLOAD_LEN) {
+      throw new DlAbortEx(EX_TOO_LONG_PAYLOAD, payloadLength);
     }
     currentPayloadLength = payloadLength;
   }
@@ -103,13 +99,13 @@ bool PeerConnection::receiveMessage(unsigned char* data, int32_t& dataLength) {
   // we have currentPayloadLen-resbufLen bytes to read
   int32_t remaining = currentPayloadLength-resbufLength;
   if(remaining > 0) {
-    socket->readData((char*)resbuf+resbufLength, remaining);
+    socket->readData(resbuf+resbufLength, remaining);
     if(remaining == 0) {
       // we got EOF
       throw new DlAbortEx(EX_EOF_FROM_PEER);
     }
     resbufLength += remaining;
-    if(currentPayloadLength != resbufLength) {
+    if(currentPayloadLength > resbufLength) {
       return false;
     }
   }
@@ -124,27 +120,23 @@ bool PeerConnection::receiveMessage(unsigned char* data, int32_t& dataLength) {
 
 bool PeerConnection::receiveHandshake(unsigned char* data, int32_t& dataLength,
 				      bool peek) {
-  int32_t remain = BtHandshakeMessage::MESSAGE_LENGTH-resbufLength;
-  if(remain != 0 && !socket->isReadable(0)) {
+  int32_t remaining = BtHandshakeMessage::MESSAGE_LENGTH-resbufLength;
+  if(remaining > 0 && !socket->isReadable(0)) {
     dataLength = 0;
     return false;
   }
-  int32_t temp = remain;
-  if(remain != 0) {
-    socket->readData((char*)resbuf+resbufLength, temp);
-    if(temp == 0) {
+  bool retval = true;
+  if(remaining > 0) {
+    socket->readData(resbuf+resbufLength, remaining);
+    if(remaining == 0) {
       // we got EOF
       throw new DlAbortEx(EX_EOF_FROM_PEER);
     }
+    resbufLength += remaining;
+    if(BtHandshakeMessage::MESSAGE_LENGTH > resbufLength) {
+      retval = false;
+    }
   }
-  bool retval;
-  if(remain != temp) {
-    retval = false;
-  } else {
-    retval = true;
-  }
-  resbufLength += temp;
-
   int32_t writeLength = resbufLength > dataLength ? dataLength : resbufLength;
   memcpy(data, resbuf, writeLength);
   dataLength = writeLength;

src/PeerConnection.h

@@ -54,7 +54,7 @@ private:
   char resbuf[MAX_PAYLOAD_LEN];
   int32_t resbufLength;
   int32_t currentPayloadLength;
-  unsigned char lenbuf[4];
+  char lenbuf[4];
   int32_t lenbufLength;
 
 public:

src/PeerMessageUtil.cc

@@ -73,10 +73,6 @@ void PeerMessageUtil::checkLength(int32_t length) {
   if(length <= 0) {
     throw new DlAbortEx("Invalid length: %d", length);
   }
-  if(!Util::isPowerOf(length, 2)) {
-    throw new DlAbortEx("Invalid length: %d It is not power of 2",
-			length);
-  }
 }
 
 void PeerMessageUtil::checkRange(int32_t begin, int32_t length, int32_t pieceLength) {

src/Piece.cc

@@ -105,6 +105,7 @@ string Piece::toString() const {
 
 void Piece::reconfigure(int32_t length)
 {
+  delete bitfield;
   this->length = length;
   bitfield =
     BitfieldManFactory::getFactoryInstance()->createBitfieldMan(_blockLength, length);

src/RequestGroup.cc

@@ -33,6 +33,7 @@
  */
 /* copyright --> */
 #include "RequestGroup.h"
+#include "PostDownloadHandler.h"
 #include "DownloadEngine.h"
 #include "DefaultSegmentManFactory.h"
 #include "NullProgressInfoFile.h"

src/RequestGroupMan.cc

@@ -223,7 +223,6 @@ void RequestGroupMan::showDownloadResults(ostream& o) const
   // ===+====+=======================================================================
   o << "\n"
     <<_("Download Results:") << "\n"
-    << " (OK):download completed.(ERR):error occurred.(INPR):download in-progress." << "\n"
     << "gid|stat|path/URI" << "\n"
     << "===+====+======================================================================" << "\n";
   for(DownloadResults::const_iterator itr = _downloadResults.begin();
@@ -237,6 +236,9 @@ void RequestGroupMan::showDownloadResults(ostream& o) const
     string status = result->result == DownloadResult::FINISHED ? "OK" : "INPR";
     o << formatDownloadResult(status, result) << "\n";
   }
+  o << "\n"
+    << _("Status Legend:") << "\n"
+    << " (OK):download completed.(ERR):error occurred.(INPR):download in-progress." << "\n";
 }
 
 string RequestGroupMan::formatDownloadResult(const string& status, const DownloadResultHandle& downloadResult) const

src/SegmentMan.cc

@@ -204,10 +204,24 @@ void SegmentMan::cancelSegment(int32_t cuid) {
   }
 }
 
+class FindSegmentEntry {
+private:
+  SegmentHandle _segment;
+public:
+  FindSegmentEntry(const SegmentHandle& segment):_segment(segment) {}
+
+  bool operator()(const SegmentEntryHandle& segmentEntry) const
+  {
+    return segmentEntry->segment->getIndex() == _segment->getIndex();
+  }
+};
+
 bool SegmentMan::completeSegment(int32_t cuid, const SegmentHandle& segment) {
   _pieceStorage->completePiece(segment->getPiece());
   _pieceStorage->advertisePiece(cuid, segment->getPiece()->getIndex());
-  SegmentEntries::iterator itr = getSegmentEntryIteratorByCuid(cuid);
+  SegmentEntries::iterator itr = find_if(usedSegmentEntries.begin(),
+					 usedSegmentEntries.end(),
+					 FindSegmentEntry(segment));
   if(itr == usedSegmentEntries.end()) {
     return false;
   } else {
@@ -258,42 +272,6 @@ int32_t SegmentMan::calculateDownloadSpeed() const {
   return speed;
 }
 
-SegmentEntryHandle SegmentMan::getSegmentEntryByIndex(int32_t index)
-{
-  for(SegmentEntries::const_iterator itr = usedSegmentEntries.begin();
-      itr != usedSegmentEntries.end(); ++itr) {
-    const SegmentEntryHandle& segmentEntry = *itr;
-    if(segmentEntry->segment->getIndex() == index) {
-      return segmentEntry;
-    }
-  }
-  return 0;
-}
-  
-SegmentEntryHandle SegmentMan::getSegmentEntryByCuid(int32_t cuid)
-{
-  for(SegmentEntries::const_iterator itr = usedSegmentEntries.begin();
-      itr != usedSegmentEntries.end(); ++itr) {
-    const SegmentEntryHandle& segmentEntry = *itr;
-    if(segmentEntry->cuid == cuid) {
-      return segmentEntry;
-    }
-  }
-  return 0;    
-}
-
-SegmentEntries::iterator SegmentMan::getSegmentEntryIteratorByCuid(int32_t cuid)
-{
-  for(SegmentEntries::iterator itr = usedSegmentEntries.begin();
-      itr != usedSegmentEntries.end(); ++itr) {
-    const SegmentEntryHandle& segmentEntry = *itr;
-    if(segmentEntry->cuid == cuid) {
-      return itr;
-    }
-  }
-  return usedSegmentEntries.end();    
-}
-
 int32_t SegmentMan::countFreePieceFrom(int32_t index) const
 {
   for(int32_t i = index; i < _downloadContext->getNumPieces(); ++i) {

src/SegmentMan.h

@@ -87,13 +87,6 @@ private:
   SegmentHandle checkoutSegment(int32_t cuid, const PieceHandle& piece);
 
   SegmentEntryHandle findSlowerSegmentEntry(const PeerStatHandle& peerStat) const;
-
-  SegmentEntryHandle getSegmentEntryByIndex(int32_t index);
-  
-  SegmentEntryHandle getSegmentEntryByCuid(int32_t cuid);
-
-  SegmentEntries::iterator getSegmentEntryIteratorByCuid(int32_t cuid);
-
 public:
   SegmentMan(const Option* option,
 	     const DownloadContextHandle& downloadContext = 0,

src/StreamFileAllocationEntry.cc

@@ -54,8 +54,7 @@ StreamFileAllocationEntry::~StreamFileAllocationEntry() {}
 Commands StreamFileAllocationEntry::prepareForNextAction(DownloadEngine* e)
 {
   Commands commands;
-  if(_timer.difference() <= e->option->getAsInt(PREF_DIRECT_DOWNLOAD_TIMEOUT) &&
-     _nextCommand) {
+  if(_nextCommand) {
     commands.push_back(popNextCommand());
     // try remaining uris
     Commands streamCommands = _requestGroup->createNextCommandWithAdj(e, -1);

src/XML2SAXMetalinkProcessor.cc

@@ -146,19 +146,23 @@ MetalinkerHandle XML2SAXMetalinkProcessor::parseFromBinaryStream(const BinaryStr
 
   SessionDataHandle sessionData = new SessionData(_stm);
   xmlParserCtxtPtr ctx = xmlCreatePushParserCtxt(&mySAXHandler, sessionData.get(), (const char*)buf, res, 0);
-
-  int64_t readOffset = res;
-  while(1) {
-    int32_t res = binaryStream->readData(buf, bufSize, readOffset);
-    if(res == 0) {
-      break;
-    }
-    if(xmlParseChunk(ctx, (const char*)buf, res, 0) != 0) {
-      throw new DlAbortEx(MSG_CANNOT_PARSE_METALINK);
+  try {
+    int64_t readOffset = res;
+    while(1) {
+      int32_t res = binaryStream->readData(buf, bufSize, readOffset);
+      if(res == 0) {
+	break;
+      }
+      if(xmlParseChunk(ctx, (const char*)buf, res, 0) != 0) {
+	throw new DlAbortEx(MSG_CANNOT_PARSE_METALINK);
+      }
+      readOffset += res;
     }
-    readOffset += res;
+    xmlParseChunk(ctx, (const char*)buf, 0, 1);
+  } catch(Exception* e) {
+    xmlFreeParserCtxt(ctx);
+    throw e;
   }
-  xmlParseChunk(ctx, (const char*)buf, 0, 1);
   xmlFreeParserCtxt(ctx);
 
   if(!_stm->finished()) {

src/message.h

@@ -204,6 +204,6 @@
 #define EX_INVALID_RANGE_HEADER _("Invalid range header. Request: %s-%s/%s, Response: %s-%s/%s")
 #define EX_NO_RESULT_WITH_YOUR_PREFS _("No file matched with your preference.")
 #define EX_EXCEPTION_CAUGHT _("Exception caught")
-#define EX_TOO_LONG_PAYLOAD _("Max payload length exceeded or invalid. length = %d")
+#define EX_TOO_LONG_PAYLOAD _("Max payload length exceeded or invalid. length = %u")
 #define EX_FILE_LENGTH_MISMATCH_BETWEEN_LOCAL_AND_REMOTE _("Invalid file length. Cannot continue download %s: local %s, remote %s")
 #endif // _D_MESSAGE_H_

test/AnnounceListTest.cc

@@ -270,10 +270,13 @@ void AnnounceListTest::testMoveToStoppedAllowedTier() {
 
   AnnounceList announceList(tiers);
 
+  CPPUNIT_ASSERT(!announceList.currentTierAcceptsStoppedEvent());
   CPPUNIT_ASSERT_EQUAL(string("tracker1"), announceList.getAnnounce());
   announceList.moveToStoppedAllowedTier();
+  CPPUNIT_ASSERT(announceList.currentTierAcceptsStoppedEvent());
   CPPUNIT_ASSERT_EQUAL(string("tracker2"), announceList.getAnnounce());
   announceList.announceFailure();
+  CPPUNIT_ASSERT(!announceList.currentTierAcceptsStoppedEvent());
   CPPUNIT_ASSERT_EQUAL(string("tracker3"), announceList.getAnnounce());
   // test wrapped search
   announceList.moveToStoppedAllowedTier();
@@ -293,10 +296,13 @@ void AnnounceListTest::testMoveToCompletedAllowedTier() {
 
   AnnounceList announceList(tiers);
 
+  CPPUNIT_ASSERT(!announceList.currentTierAcceptsCompletedEvent());
   CPPUNIT_ASSERT_EQUAL(string("tracker1"), announceList.getAnnounce());
   announceList.moveToStoppedAllowedTier();
+  CPPUNIT_ASSERT(announceList.currentTierAcceptsCompletedEvent());
   CPPUNIT_ASSERT_EQUAL(string("tracker2"), announceList.getAnnounce());
   announceList.announceFailure();
+  CPPUNIT_ASSERT(!announceList.currentTierAcceptsCompletedEvent());
   CPPUNIT_ASSERT_EQUAL(string("tracker3"), announceList.getAnnounce());
   // test wrapped search
   announceList.moveToStoppedAllowedTier();

test/CookieBoxFactoryTest.cc

@@ -38,18 +38,30 @@ void CookieBoxFactoryTest::testLoadDefaultCookie()
   Cookie c = cookies[0];
   CPPUNIT_ASSERT_EQUAL(string("JSESSIONID"), c.name);
   CPPUNIT_ASSERT_EQUAL(string("123456789"), c.value);
+  CPPUNIT_ASSERT_EQUAL((time_t)1181473200, c.expires);
+  CPPUNIT_ASSERT_EQUAL(string("/"), c.path);
+  CPPUNIT_ASSERT_EQUAL(string("localhost"), c.domain);
 
   c = cookies[1];
   CPPUNIT_ASSERT_EQUAL(string("user"), c.name);
   CPPUNIT_ASSERT_EQUAL(string("me"), c.value);
+  CPPUNIT_ASSERT_EQUAL((time_t)1181473200, c.expires);
+  CPPUNIT_ASSERT_EQUAL(string("/"), c.path);
+  CPPUNIT_ASSERT_EQUAL(string("localhost"), c.domain);
 
   c = cookies[2];
   CPPUNIT_ASSERT_EQUAL(string("passwd"), c.name);
   CPPUNIT_ASSERT_EQUAL(string("secret"), c.value);
+  CPPUNIT_ASSERT_EQUAL((time_t)2147483647, c.expires);
+  CPPUNIT_ASSERT_EQUAL(string("/cgi-bin"), c.path);
+  CPPUNIT_ASSERT_EQUAL(string("localhost"), c.domain);
 
   c = cookies[3];
   CPPUNIT_ASSERT_EQUAL(string("novalue"), c.name);
   CPPUNIT_ASSERT_EQUAL(string(""), c.value);
+  CPPUNIT_ASSERT_EQUAL((time_t)2147483647, c.expires);
+  CPPUNIT_ASSERT_EQUAL(string("/"), c.path);
+  CPPUNIT_ASSERT_EQUAL(string("localhost"), c.domain);
 }
 
 void CookieBoxFactoryTest::testCreateNewInstance()
@@ -60,5 +72,5 @@ void CookieBoxFactoryTest::testCreateNewInstance()
   CookieBoxHandle box = factory.createNewInstance();
   Cookies cookies = box->criteriaFind("localhost", "/", 0, true);
 
-  CPPUNIT_ASSERT_EQUAL((int32_t)4, (int32_t)cookies.size());
+  CPPUNIT_ASSERT_EQUAL((int32_t)3, (int32_t)cookies.size());
 }

test/DefaultBtAnnounceTest.cc

@@ -19,6 +19,8 @@ class DefaultBtAnnounceTest:public CppUnit::TestFixture {
   CPPUNIT_TEST(testGetAnnounceUrl);
   CPPUNIT_TEST(testNoMoreAnnounce);
   CPPUNIT_TEST(testIsAllAnnounceFailed);
+  CPPUNIT_TEST(testURLOrderInStoppedEvent);
+  CPPUNIT_TEST(testURLOrderInCompletedEvent);
   CPPUNIT_TEST_SUITE_END();
 private:
   MockBtContextHandle _btContext;
@@ -74,6 +76,8 @@ public:
   void testGetAnnounceUrl();
   void testNoMoreAnnounce();
   void testIsAllAnnounceFailed();
+  void testURLOrderInStoppedEvent();
+  void testURLOrderInCompletedEvent();
 };
 
 
@@ -211,3 +215,63 @@ void DefaultBtAnnounceTest::testIsAllAnnounceFailed()
 
   CPPUNIT_ASSERT(!btAnnounce.isAllAnnounceFailed());  
 }
+
+void DefaultBtAnnounceTest::testURLOrderInStoppedEvent()
+{
+  const char* urls[] = { "http://localhost1/announce",
+			 "http://localhost2/announce" };
+  AnnounceTierHandle announceTier = new AnnounceTier(Strings(&urls[0], &urls[2]));
+
+  _btContext->addAnnounceTier(announceTier);
+
+  DefaultBtAnnounce btAnnounce(_btContext, _option);
+  btAnnounce.setPieceStorage(_pieceStorage);
+  btAnnounce.setPeerStorage(_peerStorage);
+  btAnnounce.setBtRuntime(_btRuntime);
+  btAnnounce.setRandomizer(new FixedNumberRandomizer());
+  btAnnounce.generateKey();
+
+  CPPUNIT_ASSERT_EQUAL(string("http://localhost1/announce?info_hash=%01%23Eg%89%ab%cd%ef%01%23Eg%89%ab%cd%ef%01%23Eg&peer_id=%2daria2%2dultrafastdltl&uploaded=1572864&downloaded=1310720&left=1572864&compact=1&key=AAAAAAAA&numwant=50&no_peer_id=1&port=6989&event=started"), btAnnounce.getAnnounceUrl());
+
+  btAnnounce.announceSuccess();
+
+  _btRuntime->setHalt(true);
+
+  CPPUNIT_ASSERT_EQUAL(string("http://localhost1/announce?info_hash=%01%23Eg%89%ab%cd%ef%01%23Eg%89%ab%cd%ef%01%23Eg&peer_id=%2daria2%2dultrafastdltl&uploaded=1572864&downloaded=1310720&left=1572864&compact=1&key=AAAAAAAA&numwant=0&no_peer_id=1&port=6989&event=stopped"), btAnnounce.getAnnounceUrl());
+
+  btAnnounce.announceFailure();
+
+  CPPUNIT_ASSERT_EQUAL(string("http://localhost2/announce?info_hash=%01%23Eg%89%ab%cd%ef%01%23Eg%89%ab%cd%ef%01%23Eg&peer_id=%2daria2%2dultrafastdltl&uploaded=1572864&downloaded=1310720&left=1572864&compact=1&key=AAAAAAAA&numwant=0&no_peer_id=1&port=6989&event=stopped"), btAnnounce.getAnnounceUrl());
+
+  btAnnounce.announceSuccess();
+}
+
+void DefaultBtAnnounceTest::testURLOrderInCompletedEvent()
+{
+  const char* urls[] = { "http://localhost1/announce",
+			 "http://localhost2/announce" };
+  AnnounceTierHandle announceTier = new AnnounceTier(Strings(&urls[0], &urls[2]));
+
+  _btContext->addAnnounceTier(announceTier);
+
+  DefaultBtAnnounce btAnnounce(_btContext, _option);
+  btAnnounce.setPieceStorage(_pieceStorage);
+  btAnnounce.setPeerStorage(_peerStorage);
+  btAnnounce.setBtRuntime(_btRuntime);
+  btAnnounce.setRandomizer(new FixedNumberRandomizer());
+  btAnnounce.generateKey();
+
+  CPPUNIT_ASSERT_EQUAL(string("http://localhost1/announce?info_hash=%01%23Eg%89%ab%cd%ef%01%23Eg%89%ab%cd%ef%01%23Eg&peer_id=%2daria2%2dultrafastdltl&uploaded=1572864&downloaded=1310720&left=1572864&compact=1&key=AAAAAAAA&numwant=50&no_peer_id=1&port=6989&event=started"), btAnnounce.getAnnounceUrl());
+
+  btAnnounce.announceSuccess();
+
+  _pieceStorage->setAllDownloadFinished(true);
+
+  CPPUNIT_ASSERT_EQUAL(string("http://localhost1/announce?info_hash=%01%23Eg%89%ab%cd%ef%01%23Eg%89%ab%cd%ef%01%23Eg&peer_id=%2daria2%2dultrafastdltl&uploaded=1572864&downloaded=1310720&left=1572864&compact=1&key=AAAAAAAA&numwant=50&no_peer_id=1&port=6989&event=completed"), btAnnounce.getAnnounceUrl());
+
+  btAnnounce.announceFailure();
+
+  CPPUNIT_ASSERT_EQUAL(string("http://localhost2/announce?info_hash=%01%23Eg%89%ab%cd%ef%01%23Eg%89%ab%cd%ef%01%23Eg&peer_id=%2daria2%2dultrafastdltl&uploaded=1572864&downloaded=1310720&left=1572864&compact=1&key=AAAAAAAA&numwant=50&no_peer_id=1&port=6989&event=completed"), btAnnounce.getAnnounceUrl());
+
+  btAnnounce.announceSuccess();
+}

test/Metalink2RequestGroupTest.cc

@@ -33,6 +33,7 @@ void Metalink2RequestGroupTest::testGenerate()
   {
     RequestGroupHandle rg = groups[0];
     Strings uris = rg->getUris();
+    sort(uris.begin(), uris.end());
     CPPUNIT_ASSERT_EQUAL((size_t)2, uris.size());
     CPPUNIT_ASSERT_EQUAL(string("ftp://ftphost/aria2-0.5.2.tar.bz2"), uris[0]);
     CPPUNIT_ASSERT_EQUAL(string("http://httphost/aria2-0.5.2.tar.bz2"), uris[1]);

test/SegmentManTest.cc

@@ -4,8 +4,10 @@
 #include "Util.h"
 #include "SingleFileDownloadContext.h"
 #include "UnknownLengthPieceStorage.h"
+#include "DefaultPieceStorage.h"
 #include "Segment.h"
 #include "Option.h"
+#include "MockBtContext.h"
 #include <cppunit/extensions/HelperMacros.h>
 
 using namespace std;
@@ -14,6 +16,7 @@ class SegmentManTest:public CppUnit::TestFixture {
 
   CPPUNIT_TEST_SUITE(SegmentManTest);
   CPPUNIT_TEST(testNullBitfield);
+  CPPUNIT_TEST(testCompleteSegment);
   CPPUNIT_TEST(testMarkPieceDone_usedSegment);
   CPPUNIT_TEST_SUITE_END();
 private:
@@ -23,6 +26,7 @@ public:
   }
 
   void testNullBitfield();
+  void testCompleteSegment();
   void testMarkPieceDone_usedSegment();
 };
 
@@ -50,6 +54,33 @@ void SegmentManTest::testNullBitfield()
   CPPUNIT_ASSERT(!segmentMan.getSegment(2).isNull());
 }
 
+void SegmentManTest::testCompleteSegment()
+{
+  Option op;
+  int32_t pieceLength = 1024*1024;
+  int64_t totalLength = 64*1024*1024;
+  MockBtContextHandle dctx = new MockBtContext();
+  dctx->setPieceLength(pieceLength);
+  dctx->setTotalLength(totalLength);
+  dctx->setNumPieces((totalLength+pieceLength-1)/pieceLength);
+  DefaultPieceStorageHandle ps = new DefaultPieceStorage(dctx, &op);
+
+  SegmentMan segmentMan(&op, dctx, ps);
+
+  CPPUNIT_ASSERT(!segmentMan.getSegment(1, 0).isNull());
+  SegmentHandle seg = segmentMan.getSegment(1, 1);
+  CPPUNIT_ASSERT(!seg.isNull());
+  CPPUNIT_ASSERT(!segmentMan.getSegment(1, 2).isNull());
+
+  seg->updateWrittenLength(pieceLength);
+  segmentMan.completeSegment(1, seg);
+  
+  Segments segments = segmentMan.getInFlightSegment(1);
+  CPPUNIT_ASSERT_EQUAL((size_t)2, segments.size());
+  CPPUNIT_ASSERT_EQUAL(0, segments[0]->getIndex());
+  CPPUNIT_ASSERT_EQUAL(2, segments[1]->getIndex());
+}
+
 void SegmentManTest::testMarkPieceDone_usedSegment()
 {
   // TODO implement this later

test/nscookietest.txt

@@ -3,5 +3,6 @@ localhost	FALSE	/	TRUE	1181473200	JSESSIONID	123456789
 
 localhost	FALSE	/	FALSE	1181473200	user	me
 
-localhost	FALSE	/	FALSE	1181473200	passwd	secret
-localhost	FALSE	/	FALSE	1181473200	novalue	
+localhost	FALSE	/cgi-bin	FALSE	2147483647	passwd	secret
+overflow	FALSE	/	FALSE	FALSE	9223372036854775807	TAX	1000
+localhost	FALSE	/	FALSE	2147483648	novalue