Difference between revisions of "Improve the Vault (GSoC task)"

From Software Heritage Wiki
Jump to: navigation, search
Line 1: Line 1:
 +
== Introduction ==
 +
 
The Software Heritage archive allows retrieval of archived objects of different formats.
 
The Software Heritage archive allows retrieval of archived objects of different formats.
 
Once an object has been chosen for retrieval, it can be "cooked" using the [https://docs.softwareheritage.org/devel/swh-vault/index.html Software Heritage Vault].
 
Once an object has been chosen for retrieval, it can be "cooked" using the [https://docs.softwareheritage.org/devel/swh-vault/index.html Software Heritage Vault].
 +
 +
== Task description ==
  
 
Right now the Vault has several limitations: it only handles two kinds of objects (revisions and directories), it requires recursively requesting the database to get the full subgraph of an object, and it generates revisions in an unpractical format (git fast-import).
 
Right now the Vault has several limitations: it only handles two kinds of objects (revisions and directories), it requires recursively requesting the database to get the full subgraph of an object, and it generates revisions in an unpractical format (git fast-import).

Revision as of 12:27, 10 February 2021

Introduction

The Software Heritage archive allows retrieval of archived objects of different formats. Once an object has been chosen for retrieval, it can be "cooked" using the Software Heritage Vault.

Task description

Right now the Vault has several limitations: it only handles two kinds of objects (revisions and directories), it requires recursively requesting the database to get the full subgraph of an object, and it generates revisions in an unpractical format (git fast-import).

Several improvements are possible:

  • add coverage for new kinds of objects (releases, snapshots and even origins?)
  • use our in-memory graph database swh-graph to speed up fetching the necessary subgraphs.
  • write cookers to output new formats (e.g git tarballs/git bundles or even other VCS?)
  • improve unit and end-to-end testing
  • other general code improvements (better progress/error reporting in the frontend, etc.)