Difference between revisions of "Google Summer of Code 2019/Increase archive coverage"
Jump to navigation
Jump to search
| (One intermediate revision by one other user not shown) | |||
| Line 11: | Line 11: | ||
Archit Agrawal | Archit Agrawal | ||
* [https://forge.softwareheritage.org/p/nahimilega/ Forge activity] | * [https://forge.softwareheritage.org/p/nahimilega/ Forge activity] | ||
| + | * [https://wiki.softwareheritage.org/wiki/Google_Summer_of_Code_2019/Increase_archive_coverage/Commit_list List Of Commits] | ||
=== Mentors:=== | === Mentors:=== | ||
| Line 51: | Line 52: | ||
*Multiple language integration in a python library (Used in CRAN Lister) | *Multiple language integration in a python library (Used in CRAN Lister) | ||
*Different programming methodologies explained to me by my mentors(eg [https://en.wikipedia.org/wiki/Test-driven_development TDD]) | *Different programming methodologies explained to me by my mentors(eg [https://en.wikipedia.org/wiki/Test-driven_development TDD]) | ||
| − | *Work with tools; DVCS (git), issue tracker (phabricator forge), docker | + | *Work with tools; DVCS (git), issue tracker (phabricator forge), containerization/virtualization (docker) |
=== Activity reports:=== | === Activity reports:=== | ||
Latest revision as of 21:46, 28 August 2019
Title:
Increase archive coverage
Description:
The goal of this project is to increase the archive coverage by making listers and loaders for different forges. Listers are components that crawl the APIs of software forges (e.g., Bitbucket, GitHub, Sourceforge, ...) and return a list of the software available in it. Loaders take a bundle of software (tarball, Git repository ...) and load it into Software Heritage, by adapting it so that it matches the archive data model.
Student:
Archit Agrawal
Mentors:
- Nicolas Dandrimont
- Antoine R. Dumont
Work Done:
- Listers:
- Completed and merged
- Did research on the methods that could be used to make following listers and made an implementation plan for the same
- Add tests to PyPI Lister
- Refactor base tests for listers
- Add documentation on *How to run a new lister*
- Loaders:
- Base Package Manager Loader
- Ingesting source code from package managers is a process somewhat similar for all of the package managers. This calls for a common base implementation for loading content from package managers into the archive. I worked on this idea, analysed the steps required to make a loader and the implementation of present package manager loader. Came up with the plan to implement the base loader and made the pass(D1694, D1810, D1811, D1812, D1813, D1814, D1744). However, after the recommendation from my mentor, we changed the approach to make the base loader. Instead of making the whole base loader in one go, we decided to break it into multiple steps(3 steps) and follow the incremental approach.
- GNU Loader
- As part of the first step towards the implementation of Base Loader, GNU Loader was implemented.
- Base Package Manager Loader
TO-DO:
- Implement the Listers using the research done and the implementation plan made for Launchpad, Rubygem.
- Find the workarounds to solve the challenges in making the Maven and NuGET(.NET) Lister.
- Work on the remaining steps in order to complete the Base Package Manager Loader.
Learnings:
Working in Software Heritage was a wholesome experience. I got to learn a new thing almost every day. Here is a few of the most prominent ones:
- Work on a huge codebase
- Plan and design before jumping to code
- Write clean and well-commented code
- Learn the difference between doing projects in college and in the industry(Spoiler Alert: A lot)
- Multiple language integration in a python library (Used in CRAN Lister)
- Different programming methodologies explained to me by my mentors(eg TDD)
- Work with tools; DVCS (git), issue tracker (phabricator forge), containerization/virtualization (docker)
Activity reports:
- May 2019
- June 2019
- July 2019
- August 2019