technical-notes.rst 6.4 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160
  1. Technical Notes
  2. ===============
  3. This document describes additional technical information of aria2. The
  4. expected audience is developers.
  5. Control File (*.aria2) Format
  6. -----------------------------
  7. The control file uses a binary format to store progress information of
  8. a download. Here is the diagram for each field::
  9. 0 1 2 3
  10. 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  11. +---+-------+-------+-------------------------------------------+
  12. |VER| EXT |INFO |INFO HASH ... |
  13. |(2)| (4) |HASH | (INFO HASH LENGTH) |
  14. | | |LENGTH | |
  15. | | | (4) | |
  16. +---+---+---+-------+---+---------------+-------+---------------+
  17. |PIECE |TOTAL LENGTH |UPLOAD LENGTH |BIT- |BITFIELD ... |
  18. |LENGTH | (8) | (8) |FIELD | (BITFIELD |
  19. | (4) | | |LENGTH | LENGTH) |
  20. | | | | (4) | |
  21. +-------+-------+-------+-------+-------+-------+---------------+
  22. |NUM |INDEX |LENGTH |PIECE |PIECE BITFIELD ... |
  23. |IN- | (4) | (4) |BIT- | (PIECE BITFIELD LENGTH) |
  24. |FLIGHT | | |FIELD | |
  25. |PIECE | | |LENGTH | |
  26. | (4) | | | (4) | |
  27. +-------+-------+-------+-------+-------------------------------+
  28. ^ ^
  29. | |
  30. +-------------------------------------------------------+
  31. Repeated in (NUM IN-FLIGHT) PIECE times
  32. ``VER`` (VERSION): 2 bytes
  33. Should be either version 0(0x0000) or version 1(0x0001). In
  34. version 1, all multi-byte integers are saved in network byte
  35. order(big endian). In version 0, all multi-byte integers are saved
  36. in host byte order. aria2 1.4.1 can read both versions and only
  37. writes a control file in version 1 format. version 0 support will
  38. be disappear in the future version.
  39. ``EXT`` (EXTENSION): 4 bytes
  40. If LSB is 1(i.e. ``EXT[3]&1 == 1``), aria2 checks whether the saved
  41. !InfoHash and current downloading one are the same. If they are not
  42. the same, an exception is thrown. This is called "infoHashCheck"
  43. extension.
  44. ``INFO HASH LENGTH``: 4 bytes
  45. The length of InfoHash that is located after this field. If
  46. "infoHashCheck" extension is enabled, if this value is 0, then an
  47. exception is thrown. For http/ftp downloads, this value should be
  48. 0.
  49. ``INFO HASH``: ``(INFO HASH LENGTH)`` bytes
  50. BitTorrent InfoHash.
  51. ``PIECE LENGTH``: 4 bytes
  52. The length of the piece.
  53. ``TOTAL LENGTH``: 8 bytes
  54. The total length of the download.
  55. ``UPLOAD LENGTH``: 8 bytes
  56. The uploaded length in this download.
  57. ``BITFIELD LENGTH``: 4 bytes
  58. The length of bitfield.
  59. ``BITFIELD``: ``(BITFIELD LENGTH)`` bytes
  60. This is the bitfield which represents current download progress.
  61. ``NUM IN-FLIGHT PIECE``: 4 bytes
  62. The number of in-flight pieces. These piece is not marked
  63. 'downloaded' in the bitfield, but it has at least one downloaded
  64. chunk.
  65. The following 4 fields are repeated in ``(NUM IN-FLIGHT PIECE)``
  66. times.
  67. ``INDEX``: 4 bytes
  68. The index of the piece.
  69. ``LENGTH``: 4 bytes
  70. The length of the piece.
  71. ``PIECE BITFIELD LENGTH``: 4 bytes
  72. The length of bitfield of this piece.
  73. ``PIECE BITFIELD``: ``(PIECE BITFIELD LENGTH)`` bytes
  74. The bitfield of this piece. The each bit represents 16KiB chunk.
  75. DHT routing table file format
  76. -----------------------------
  77. aria2 saves IPv4 DHT routing table in ``${HOME}/.aria2/dht.dat`` and
  78. IPv6 DHT routing table in ``${HOME}/.aria2/dht6.dat`` by default.
  79. ``dht.dat`` and ``dht6.dat`` files use same binary encoding and have
  80. following fields. All multi byte integers are in network byte
  81. order. ``RSV`` (RESERVED) fields are reserved for future use. For now
  82. they should be all zeros::
  83. 0 1 2 3
  84. 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  85. +---+-+---+-----+---------------+---------------+---------------+
  86. |MGC|F|VER| RSV | MTIME | RSV |LOCAL NODE ID :
  87. |(2)|M|(2)| (3) | (8) | (8) | (20) :
  88. | |T| | | | | :
  89. +---+-+---+-----+-------+-------+-------+-------+---------------+
  90. :LOCAL NODE ID | RSV | NUM | RSV |
  91. : (continued) | (4) | NODE | (4) |
  92. : | | (4) | |
  93. +-+-------------+-------+-------+-+-----+-------+---------------+
  94. |P| RSV |COMPACT PEER INFO| RSV | <-+
  95. |L| (7) | (PLEN) | (24 - PLEN) | |
  96. |E| | | | |
  97. |N| | | | |
  98. +-+-------------+-----------------+-----+-------+---------------+ |
  99. | NODE ID | RSV | |
  100. | (20) | (4) | <-----------------+
  101. +---------------------------------------+-------+ Repeated in
  102. (NUM NODE) times.
  103. ``MGC`` (MAGIC): 2 bytes
  104. It must be ``0xa1 0xa2``.
  105. ``FMT`` (FORMAT ID): 1 byte
  106. The format ID should be ``0x02``.
  107. ``VER`` (VERSION): 2 bytes
  108. The version number should be ``0x00 0x03``.
  109. ``MTIME``: 8 bytes
  110. This is the time when aria2 saved the file. The value is the time
  111. since the Epoch(1970/1/1 00:00:00) in 64 bits integer.
  112. ``LOCALNODE ID``: 20 bytes
  113. Node ID of the client.
  114. ``NUM NODE``: 4 bytes
  115. The number of nodes the routing table has. ``NUM NODE`` node
  116. information follows.
  117. The data of ``NUM NODE`` node will follow. The node information are
  118. stored in the following fields. They are repeated in ``NUM NODE``
  119. times.
  120. ``PLEN`` (COMPACT PEER INFO LENGTH): 1 byte
  121. The length of compact peer info. For IPv4 DHT, it must be 6. For
  122. IPv6 DHT, it must be 18.
  123. ``COMPACT PEER INFO``: ``(PLEN)`` bytes
  124. The address and port of peer in compact peer format.
  125. ``NODE ID``: 20 bytes
  126. The node ID of this node.