Software Heritage Wiki - User contributions [en]

Google Summer of Code 2019/Increase archive coverage

2019-08-28T21:46:54Z

Archit agrawal: /* Student: */

===Title:===
'''Increase archive coverage'''

=== Description:===

The goal of this project is to increase the archive coverage by making listers and loaders for different forges.
[https://docs.softwareheritage.org/devel/swh-lister/index.html#swh-lister Listers] are components that crawl the APIs of software forges (e.g., Bitbucket, GitHub, Sourceforge, ...) and return a list of the software available in it. Loaders take a bundle of software (tarball, Git repository ...) and load it into Software Heritage, by adapting it so that it matches the archive data model.

===Student: ===
Archit Agrawal
* [https://forge.softwareheritage.org/p/nahimilega/ Forge activity]
* [https://wiki.softwareheritage.org/wiki/Google_Summer_of_Code_2019/Increase_archive_coverage/Commit_list List Of Commits]

=== Mentors:===
* Nicolas Dandrimont
* Antoine R. Dumont

===Work Done:===
* '''Listers:'''
** Completed and merged
*** [https://forge.softwareheritage.org/rDLSfedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6 Phabricator Lister]
*** [https://forge.softwareheritage.org/D1482 GNU Lister]
*** [https://forge.softwareheritage.org/rDLSa9a37a85bf9efac416cfdd152588bf01b7a063b2 CRAN Lister]
*** [https://forge.softwareheritage.org/D1584 Packagist Lister]
*** [https://forge.softwareheritage.org/D1610 CGit Lister]
** Did research on the methods that could be used to make following listers and made an implementation plan for the same
*** [https://forge.softwareheritage.org/T1734 Launchpad Lister]
*** [https://forge.softwareheritage.org/T1777 Rubygem Lister]
*** [https://forge.softwareheritage.org/T1718 NuGET(.NET) Lister]
*** [https://forge.softwareheritage.org/T1724 Maven Lister]
** [https://forge.softwareheritage.org/rDLS08ade29e6de0616a3964360454ab52b58c082b75 Add tests to PyPI Lister]
** [https://forge.softwareheritage.org/rDLSf424f07c7e628eb7a19d25f4fdb749682d97a21f Refactor base tests for listers]
** [https://forge.softwareheritage.org/D1441 Add documentation on *How to run a new lister*]
* '''Loaders:'''
** '''[https://forge.softwareheritage.org/T1389 Base Package Manager Loader]'''
*** Ingesting source code from package managers is a process somewhat similar for all of the package managers. This calls for a common base implementation for loading content from package managers into the archive. I worked on this idea, analysed the steps required to make a loader and the implementation of present package manager loader. Came up with the plan to implement the base loader and made the pass([https://forge.softwareheritage.org/D1694 D1694], [https://forge.softwareheritage.org/D1810 D1810], [https://forge.softwareheritage.org/D1811 D1811], [https://forge.softwareheritage.org/D1812 D1812], [https://forge.softwareheritage.org/D1813 D1813], [https://forge.softwareheritage.org/D1814 D1814], [https://forge.softwareheritage.org/D1744 D1744]). However, after the recommendation from my mentor, we changed the approach to make the base loader. Instead of making the whole base loader in one go, we decided to break it into multiple steps(3 steps) and follow the incremental approach.
**'''[https://forge.softwareheritage.org/D1824 GNU Loader]'''
*** As part of the first step towards the implementation of Base Loader, GNU Loader was implemented.

===TO-DO:===
* Implement the Listers using the research done and the implementation plan made for Launchpad, Rubygem.
* Find the workarounds to solve the challenges in making the Maven and NuGET(.NET) Lister.
* Work on the remaining steps in order to complete the Base Package Manager Loader.

=== Learnings: ===
Working in Software Heritage was a wholesome experience. I got to learn a new thing almost every day. Here is a few of the most prominent ones:
*Work on a huge codebase
*Plan and design before jumping to code
*Write clean and well-commented code
*Learn the difference between doing projects in college and in the industry(Spoiler Alert: '''A lot''')
*Multiple language integration in a python library (Used in CRAN Lister)
*Different programming methodologies explained to me by my mentors(eg [https://en.wikipedia.org/wiki/Test-driven_development TDD])
*Work with tools; DVCS (git), issue tracker (phabricator forge), containerization/virtualization (docker)

=== Activity reports:===
* May 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00003.html Week 20 Second Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00010.html Week 21 Third Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00017.html Week 22 First Week (Coding)]
* June 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00009.html Week 23 Second Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00016.html Week 24 Third Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00026.html Week 25 Fourth Week (Coding)(Work Summary)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00033.html Week 26 Fifth Week (First Evaluation)]
* July 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00003.html Week 27 Sixth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00006.html Week 28 Seventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00011.html Week 29 Eight Week (Coding)(Work Summary)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00015.html Week 30 Nineth Week (Second Evaluation)]
* August 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00002.html Week 31 Tenth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00004.html Week 32 Eleventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 33 Twelfth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00011.html Week 34 Thirteenth Week (Final Evaluation)]

== Links ==
* [https://forge.softwareheritage.org/source/swh-lister/ Lister source code repository]
* [https://forge.softwareheritage.org/source/swh-loader-core/ Loader source code repository]
* see project [https://summerofcode.withgoogle.com/projects/#5658995887439872 on the GSoC portal]

[[Category: Google Summer of Code]]
[[Category: Google Summer of Code 2019]]

Google Summer of Code 2019/Increase archive coverage/Commit list

2019-08-28T21:45:31Z

Archit agrawal: Created page with "Phabricator Lister: * [https://forge.softwareheritage.org/rDLSfedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6 fedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6] GNU Lister: * [https://forge...."

Phabricator Lister:
* [https://forge.softwareheritage.org/rDLSfedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6 fedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6]

GNU Lister:
* [https://forge.softwareheritage.org/rDLS151f6cd2235cedd420174ffad01fc8030ee4a787 151f6cd2235cedd420174ffad01fc8030ee4a787]
* [https://forge.softwareheritage.org/rDLSebdb959823bc004ff1766e97a409561dc48dcf3d ebdb959823bc004ff1766e97a409561dc48dcf3d]
* [https://forge.softwareheritage.org/rDLS709ba8a6e55c630f6ba7ba17595c11a2584bacda 709ba8a6e55c630f6ba7ba17595c11a2584bacda]
* [https://forge.softwareheritage.org/rDLS7c6245e663e074b95cc3422b10bd02a4d7b707db 7c6245e663e074b95cc3422b10bd02a4d7b707db]
* [https://forge.softwareheritage.org/rDLSf76b96b825f9e308d2d37b9e26bf1f13c1f916c8 f76b96b825f9e308d2d37b9e26bf1f13c1f916c8]

CRAN Lister
* [https://forge.softwareheritage.org/rDLSa9a37a85bf9efac416cfdd152588bf01b7a063b2 a9a37a85bf9efac416cfdd152588bf01b7a063b2]
* [https://forge.softwareheritage.org/rDLS5ea9d5ed392a12dc5558fe165b1cbf2c0dfbcbf0 5ea9d5ed392a12dc5558fe165b1cbf2c0dfbcbf0]

Packagist Lister
* [https://forge.softwareheritage.org/rDLS5727f15cf32345101bd38423b14ea199f6eb52e9 5727f15cf32345101bd38423b14ea199f6eb52e9]

CGit Lister
* [https://forge.softwareheritage.org/rDLSb972a2a88d25a5c63d6cbc39564c91f8062f9835 b972a2a88d25a5c63d6cbc39564c91f8062f9835]
* [https://forge.softwareheritage.org/rDLS7e3c79bb1d18e49ac446159066de33470528b809 7e3c79bb1d18e49ac446159066de33470528b809]
* [https://forge.softwareheritage.org/rDLS0bf24469b7e0ea2a3e5e9ea90cbfe690d1afb1d9 0bf24469b7e0ea2a3e5e9ea90cbfe690d1afb1d9]

GNU Loader
* [https://forge.softwareheritage.org/rDLDBASE054ef894d0ae0842ee01e573c59b90ac8c2994f7 054ef894d0ae0842ee01e573c59b90ac8c2994f7]
* [https://forge.softwareheritage.org/rDLDBASEcb253bdae866cab9b593e3de2d8701a3b4280010 cb253bdae866cab9b593e3de2d8701a3b4280010]

Miscellaneous
* [https://forge.softwareheritage.org/rCDFD179ce3202a58abeb3c88c8166169045265c41e3f 179ce3202a58abeb3c88c8166169045265c41e3f]
* [https://forge.softwareheritage.org/rCDFDdee708ec4a79dc6a7a2db4788d7ea2d40c0a309d dee708ec4a79dc6a7a2db4788d7ea2d40c0a309d]
* [https://forge.softwareheritage.org/rCDFD560df3bc13c8853b94e646a00177ccd9c6784ada 560df3bc13c8853b94e646a00177ccd9c6784ada]
* [https://forge.softwareheritage.org/rDLSb2c6ddc35b6b126990099d923e91a365727aeccb b2c6ddc35b6b126990099d923e91a365727aeccb]
* [https://forge.softwareheritage.org/rDLS8d1b5d2d2d92d305a8aa019e119a7260adb78c55 8d1b5d2d2d92d305a8aa019e119a7260adb78c55]
* [https://forge.softwareheritage.org/rDLSf424f07c7e628eb7a19d25f4fdb749682d97a21f f424f07c7e628eb7a19d25f4fdb749682d97a21f]
* [https://forge.softwareheritage.org/rDLS08ade29e6de0616a3964360454ab52b58c082b75 08ade29e6de0616a3964360454ab52b58c082b75]
* [https://forge.softwareheritage.org/rDLSf8a2ae866bc2115e1c0589f1a97902447260e7fd f8a2ae866bc2115e1c0589f1a97902447260e7fd]

Google Summer of Code 2019/Increase archive coverage

2019-08-26T13:09:04Z

Archit agrawal: /* Description: */

===Title:===
'''Increase archive coverage'''

=== Description:===

The goal of this project is to increase the archive coverage by making listers and loaders for different forges.
[https://docs.softwareheritage.org/devel/swh-lister/index.html#swh-lister Listers] are components that crawl the APIs of software forges (e.g., Bitbucket, GitHub, Sourceforge, ...) and return a list of the software available in it. Loaders take a bundle of software (tarball, Git repository ...) and load it into Software Heritage, by adapting it so that it matches the archive data model.

===Student: ===
Archit Agrawal
* [https://forge.softwareheritage.org/p/nahimilega/ Forge activity]

=== Mentors:===
* Nicolas Dandrimont
* Antoine R. Dumont

===Work Done:===
* '''Listers:'''
** Completed and merged
*** [https://forge.softwareheritage.org/rDLSfedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6 Phabricator Lister]
*** [https://forge.softwareheritage.org/D1482 GNU Lister]
*** [https://forge.softwareheritage.org/rDLSa9a37a85bf9efac416cfdd152588bf01b7a063b2 CRAN Lister]
*** [https://forge.softwareheritage.org/D1584 Packagist Lister]
*** [https://forge.softwareheritage.org/D1610 CGit Lister]
** Did research on the methods that could be used to make following listers and made an implementation plan for the same
*** [https://forge.softwareheritage.org/T1734 Launchpad Lister]
*** [https://forge.softwareheritage.org/T1777 Rubygem Lister]
*** [https://forge.softwareheritage.org/T1718 NuGET(.NET) Lister]
*** [https://forge.softwareheritage.org/T1724 Maven Lister]
** [https://forge.softwareheritage.org/rDLS08ade29e6de0616a3964360454ab52b58c082b75 Add tests to PyPI Lister]
** [https://forge.softwareheritage.org/rDLSf424f07c7e628eb7a19d25f4fdb749682d97a21f Refactor base tests for listers]
** [https://forge.softwareheritage.org/D1441 Add documentation on *How to run a new lister*]
* '''Loaders:'''
** '''[https://forge.softwareheritage.org/T1389 Base Package Manager Loader]'''
*** Ingesting source code from package managers is a process somewhat similar for all of the package managers. This calls for a common base implementation for loading content from package managers into the archive. I worked on this idea, analysed the steps required to make a loader and the implementation of present package manager loader. Came up with the plan to implement the base loader and made the pass([https://forge.softwareheritage.org/D1694 D1694], [https://forge.softwareheritage.org/D1810 D1810], [https://forge.softwareheritage.org/D1811 D1811], [https://forge.softwareheritage.org/D1812 D1812], [https://forge.softwareheritage.org/D1813 D1813], [https://forge.softwareheritage.org/D1814 D1814], [https://forge.softwareheritage.org/D1744 D1744]). However, after the recommendation from my mentor, we changed the approach to make the base loader. Instead of making the whole base loader in one go, we decided to break it into multiple steps(3 steps) and follow the incremental approach.
**'''[https://forge.softwareheritage.org/D1824 GNU Loader]'''
*** As part of the first step towards the implementation of Base Loader, GNU Loader was implemented.

===TO-DO:===
* Implement the Listers using the research done and the implementation plan made for Launchpad, Rubygem.
* Find the workarounds to solve the challenges in making the Maven and NuGET(.NET) Lister.
* Work on the remaining steps in order to complete the Base Package Manager Loader.

=== Learnings: ===
Working in Software Heritage was a wholesome experience. I got to learn a new thing almost every day. Here is a few of the most prominent ones:
*Work on a huge codebase
*Plan and design before jumping to code
*Write clean and well-commented code
*Learn the difference between doing projects in college and in the industry(Spoiler Alert: '''A lot''')
*Multiple language integration in a python library (Used in CRAN Lister)
*Different programming methodologies explained to me by my mentors(eg [https://en.wikipedia.org/wiki/Test-driven_development TDD])
*Work with tools; DVCS (git), issue tracker (phabricator forge), docker

=== Activity reports:===
* May 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00003.html Week 20 Second Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00010.html Week 21 Third Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00017.html Week 22 First Week (Coding)]
* June 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00009.html Week 23 Second Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00016.html Week 24 Third Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00026.html Week 25 Fourth Week (Coding)(Work Summary)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00033.html Week 26 Fifth Week (First Evaluation)]
* July 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00003.html Week 27 Sixth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00006.html Week 28 Seventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00011.html Week 29 Eight Week (Coding)(Work Summary)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00015.html Week 30 Nineth Week (Second Evaluation)]
* August 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00002.html Week 31 Tenth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00004.html Week 32 Eleventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 33 Twelfth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00011.html Week 34 Thirteenth Week (Final Evaluation)]

== Links ==
* [https://forge.softwareheritage.org/source/swh-lister/ Lister source code repository]
* [https://forge.softwareheritage.org/source/swh-loader-core/ Loader source code repository]
* see project [https://summerofcode.withgoogle.com/projects/#5658995887439872 on the GSoC portal]

[[Category: Google Summer of Code]]
[[Category: Google Summer of Code 2019]]

Google Summer of Code 2019/Increase archive coverage

2019-08-24T17:58:48Z

Archit agrawal: /* Activity reports: */

===Title:===
'''Increase archive coverage'''

=== Description:===
As Software Heritage works on archiving and sharing source code, one of the major tasks is to ingest the latest source code available in the database from time to time and from all the possible sources where you can fetch the source code using listers and ingest them using loaders. [https://docs.softwareheritage.org/devel/swh-lister/index.html#swh-lister Listers] are components that crawl the APIs of software forges (e.g., Bitbucket, GitHub, Sourceforge, ...) and return a list of the software available in it whereas [Loaders take a bundle of software (tarball, Git repository ...) and load it into Software Heritage, by adapting it so that it matches the archive data model. The goal of this project is to increase the archive coverage by making listers and loaders for different websites that which stores source code, so that Software Heritage can fetch as much source code as possible and store it in the database to preserve it for future generations.

===Student: ===
Archit Agrawal
* [https://forge.softwareheritage.org/p/nahimilega/ Forge activity]

=== Mentors:===
* Nicolas Dandrimont
* Antoine R. Dumont

===Work Done:===
* '''Listers:'''
** Completed and merged
*** [https://forge.softwareheritage.org/rDLSfedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6 Phabricator Lister]
*** [https://forge.softwareheritage.org/D1482 GNU Lister]
*** [https://forge.softwareheritage.org/rDLSa9a37a85bf9efac416cfdd152588bf01b7a063b2 CRAN Lister]
*** [https://forge.softwareheritage.org/D1584 Packagist Lister]
*** [https://forge.softwareheritage.org/D1610 CGit Lister]
** Did research on the methods that could be used to make following listers and made an implementation plan for the same
*** [https://forge.softwareheritage.org/T1734 Launchpad Lister]
*** [https://forge.softwareheritage.org/T1777 Rubygem Lister]
*** [https://forge.softwareheritage.org/T1718 NuGET(.NET) Lister]
*** [https://forge.softwareheritage.org/T1724 Maven Lister]
** [https://forge.softwareheritage.org/rDLS08ade29e6de0616a3964360454ab52b58c082b75 Add tests to PyPI Lister]
** [https://forge.softwareheritage.org/rDLSf424f07c7e628eb7a19d25f4fdb749682d97a21f Refractor base tests for listers]
** [https://forge.softwareheritage.org/D1441 Add documentation on *How to run a new lister*]
* '''Loaders:'''
** '''[https://forge.softwareheritage.org/T1389 Base Package Manager Loader]'''
*** Ingesting source code from package managers is a process somewhat similar for all of the package managers. This calls for a common base implementation for loading content from package managers into the archive. I worked on this idea, analysed the steps required to make a loader and the implementation of present package manager loader. Came up with the plan to implement the base loader and made the pass([https://forge.softwareheritage.org/D1694 D1694], [https://forge.softwareheritage.org/D1810 D1810], [https://forge.softwareheritage.org/D1811 D1811], [https://forge.softwareheritage.org/D1812 D1812], [https://forge.softwareheritage.org/D1813 D1813], [https://forge.softwareheritage.org/D1814 D1814], [https://forge.softwareheritage.org/D1744 D1744]). However, after the recommendation from my mentor, we changed the approach to make the base loader. Instead of making the whole base loader in one go, we decided to break it into multiple steps(3 steps) and follow the incremental approach.
**'''[https://forge.softwareheritage.org/D1824 GNU Loader]'''
*** As part of the first step towards the implementation of Base Loader, GNU Loader was implemented.

===TO-DO:===
* Implement the Listers using the research done and the implementation plan made for Launchpad, Rubygem.
* Find the workarounds to solve the challenges in making the Maven and NuGET(.NET) Lister.
* Work on the remaining steps in order to complete the Base Package Manager Loader.

=== Learnings: ===
Working in Software Heritage was a wholesome experience. I got to learn a new thing almost every day. It would me injustice id I say I can account all my learnings in a section of a blog, however here are a list of few of most prominent once:
*Working on a huge codebase
*Plan and design before jumping to code
*Writing clean and well-commented code
*Difference between doing projects in college and in the industry(Spoiler Alert: '''A lot''')
*Multiple language integration in a python library (Used in CRAN Lister)
*Different programming methodologies explained to me by my mentors(eg [https://en.wikipedia.org/wiki/Test-driven_development TDD])
*Working with git and forge
*Working with Docker

=== Activity reports:===
* May 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00003.html Week 20 Second Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00010.html Week 21 Third Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00017.html Week 22 First Week (Coding)]
* June 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00009.html Week 23 Second Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00016.html Week 24 Third Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00026.html Week 25 Fourth Week (Coding)(Work Summary)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00033.html Week 26 Fifth Week (First Evaluation)]
* July 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00003.html Week 27 Sixth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00006.html Week 28 Seventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00011.html Week 29 Eight Week (Coding)(Work Summary)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00015.html Week 30 Nineth Week (Second Evaluation)]
* August 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00002.html Week 31 Tenth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00004.html Week 32 Eleventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 33 Twelfth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00011.html Week 34 Thirteenth Week (Final Evaluation)]

== Links ==
* [https://forge.softwareheritage.org/source/swh-lister/ Lister source code repository]
* [https://forge.softwareheritage.org/source/swh-loader-core/ Loader source code repository]
* see project [https://summerofcode.withgoogle.com/projects/#5658995887439872 on the GSoC portal]

[[Category: Google Summer of Code]]
[[Category: Google Summer of Code 2019]]

Google Summer of Code 2019/Increase archive coverage

2019-08-23T13:18:07Z

Archit agrawal:

===Title:===
'''Increase archive coverage'''

=== Description:===
As Software Heritage works on archiving and sharing source code, one of the major tasks is to ingest the latest source code available in the database from time to time and from all the possible sources where you can fetch the source code using listers and ingest them using loaders. [https://docs.softwareheritage.org/devel/swh-lister/index.html#swh-lister Listers] are components that crawl the APIs of software forges (e.g., Bitbucket, GitHub, Sourceforge, ...) and return a list of the software available in it whereas [Loaders take a bundle of software (tarball, Git repository ...) and load it into Software Heritage, by adapting it so that it matches the archive data model. The goal of this project is to increase the archive coverage by making listers and loaders for different websites that which stores source code, so that Software Heritage can fetch as much source code as possible and store it in the database to preserve it for future generations.

===Student: ===
Archit Agrawal
* [https://forge.softwareheritage.org/p/nahimilega/ Forge activity]

=== Mentors:===
* Nicolas Dandrimont
* Antoine R. Dumont

===Work Done:===
* '''Listers:'''
** Completed and merged
*** [https://forge.softwareheritage.org/rDLSfedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6 Phabricator Lister]
*** [https://forge.softwareheritage.org/D1482 GNU Lister]
*** [https://forge.softwareheritage.org/rDLSa9a37a85bf9efac416cfdd152588bf01b7a063b2 CRAN Lister]
*** [https://forge.softwareheritage.org/D1584 Packagist Lister]
*** [https://forge.softwareheritage.org/D1610 CGit Lister]
** Did research on the methods that could be used to make following listers and made an implementation plan for the same
*** [https://forge.softwareheritage.org/T1734 Launchpad Lister]
*** [https://forge.softwareheritage.org/T1777 Rubygem Lister]
*** [https://forge.softwareheritage.org/T1718 NuGET(.NET) Lister]
*** [https://forge.softwareheritage.org/T1724 Maven Lister]
** [https://forge.softwareheritage.org/rDLS08ade29e6de0616a3964360454ab52b58c082b75 Add tests to PyPI Lister]
** [https://forge.softwareheritage.org/rDLSf424f07c7e628eb7a19d25f4fdb749682d97a21f Refractor base tests for listers]
** [https://forge.softwareheritage.org/D1441 Add documentation on *How to run a new lister*]
* '''Loaders:'''
** '''[https://forge.softwareheritage.org/T1389 Base Package Manager Loader]'''
*** Ingesting source code from package managers is a process somewhat similar for all of the package managers. This calls for a common base implementation for loading content from package managers into the archive. I worked on this idea, analysed the steps required to make a loader and the implementation of present package manager loader. Came up with the plan to implement the base loader and made the pass([https://forge.softwareheritage.org/D1694 D1694], [https://forge.softwareheritage.org/D1810 D1810], [https://forge.softwareheritage.org/D1811 D1811], [https://forge.softwareheritage.org/D1812 D1812], [https://forge.softwareheritage.org/D1813 D1813], [https://forge.softwareheritage.org/D1814 D1814], [https://forge.softwareheritage.org/D1744 D1744]). However, after the recommendation from my mentor, we changed the approach to make the base loader. Instead of making the whole base loader in one go, we decided to break it into multiple steps(3 steps) and follow the incremental approach.
**'''[https://forge.softwareheritage.org/D1824 GNU Loader]'''
*** As part of the first step towards the implementation of Base Loader, GNU Loader was implemented.

===TO-DO:===
* Implement the Listers using the research done and the implementation plan made for Launchpad, Rubygem.
* Find the workarounds to solve the challenges in making the Maven and NuGET(.NET) Lister.
* Work on the remaining steps in order to complete the Base Package Manager Loader.

=== Learnings: ===
Working in Software Heritage was a wholesome experience. I got to learn a new thing almost every day. It would me injustice id I say I can account all my learnings in a section of a blog, however here are a list of few of most prominent once:
*Working on a huge codebase
*Plan and design before jumping to code
*Writing clean and well-commented code
*Difference between doing projects in college and in the industry(Spoiler Alert: '''A lot''')
*Multiple language integration in a python library (Used in CRAN Lister)
*Different programming methodologies explained to me by my mentors(eg [https://en.wikipedia.org/wiki/Test-driven_development TDD])
*Working with git and forge
*Working with Docker

=== Activity reports:===
* May 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00003.html Week 20 Second Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00010.html Week 21 Third Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00017.html Week 22 First Week (Coding)]
* June 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00009.html Week 23 Second Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00016.html Week 24 Third Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00026.html Week 25 Fourth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00033.html Week 26 Fifth Week (First Evaluation)]
* July 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00003.html Week 27 Sixth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00006.html Week 28 Seventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00011.html Week 29 Eight Week (Coding)(Work Summary)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00015.html Week 30 Nineth Week (Second Evaluation)]
* August 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00002.html Week 31 Tenth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00004.html Week 32 Eleventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 33 Twelfth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 34 Thirteenth Week (Final Evaluation)]

== Links ==
* [https://forge.softwareheritage.org/source/swh-lister/ Lister source code repository]
* [https://forge.softwareheritage.org/source/swh-loader-core/ Loader source code repository]
* see project [https://summerofcode.withgoogle.com/projects/#5658995887439872 on the GSoC portal]

[[Category: Google Summer of Code]]
[[Category: Google Summer of Code 2019]]

Google Summer of Code 2019/Increase archive coverage

2019-08-23T13:08:55Z

Archit agrawal:

===Title: ===
Increase archive coverage

=== Description:===
As Software Heritage works on archiving and sharing source code, one of the major tasks is to ingest the latest source code available in the database from time to time and from all the possible sources where you can fetch the source code using listers and ingest them using loaders. [https://docs.softwareheritage.org/devel/swh-lister/index.html#swh-lister Listers] are components that crawl the APIs of software forges (e.g., Bitbucket, GitHub, Sourceforge, ...) and return a list of the software available in it whereas [Loaders take a bundle of software (tarball, Git repository ...) and load it into Software Heritage, by adapting it so that it matches the archive data model. The goal of this project is to increase the archive coverage by making listers and loaders for different websites that which stores source code, so that Software Heritage can fetch as much source code as possible and store it in the database to preserve it for future generations.

===Student: ===
Archit Agrawal
* [https://forge.softwareheritage.org/p/nahimilega/ Forge activity]

=== Mentors:===
* Nicolas Dandrimont
* Antoine R. Dumont

===Work Done:===
* '''Listers:'''
** Completed and merged
*** [https://forge.softwareheritage.org/rDLSfedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6 Phabricator Lister]
*** [https://forge.softwareheritage.org/D1482 GNU Lister]
*** [https://forge.softwareheritage.org/rDLSa9a37a85bf9efac416cfdd152588bf01b7a063b2 CRAN Lister]
*** [https://forge.softwareheritage.org/D1584 Packagist Lister]
*** [https://forge.softwareheritage.org/D1610 CGit Lister]
** Did research on the methods that could be used to make following listers and made an implementation plan for the same
*** [https://forge.softwareheritage.org/T1734 Launchpad Lister]
*** [https://forge.softwareheritage.org/T1777 Rubygem Lister]
*** [https://forge.softwareheritage.org/T1718 NuGET(.NET) Lister]
*** [https://forge.softwareheritage.org/T1724 Maven Lister]
** [https://forge.softwareheritage.org/rDLS08ade29e6de0616a3964360454ab52b58c082b75 Add tests to PyPI Lister]
** [https://forge.softwareheritage.org/rDLSf424f07c7e628eb7a19d25f4fdb749682d97a21f Refractor base tests for listers]
** [https://forge.softwareheritage.org/D1441 Add documentation on *How to run a new lister*]
* '''Loaders:'''
** '''[https://forge.softwareheritage.org/T1389 Base Package Manager Loader]'''
*** Ingesting source code from package managers is a process somewhat similar for all of the package managers. This calls for a common base implementation for loading content from package managers into the archive. I worked on this idea, analysed the steps required to make a loader and the implementation of present package manager loader. Came up with the plan to implement the base loader and made the pass([https://forge.softwareheritage.org/D1694 D1694], [https://forge.softwareheritage.org/D1810 D1810], [https://forge.softwareheritage.org/D1811 D1811], [https://forge.softwareheritage.org/D1812 D1812], [https://forge.softwareheritage.org/D1813 D1813], [https://forge.softwareheritage.org/D1814 D1814], [https://forge.softwareheritage.org/D1744 D1744]). However, after the recommendation from my mentor, we changed the approach to make the base loader. Instead of making the whole base loader in one go, we decided to break it into multiple steps(3 steps) and follow the incremental approach.
**'''[https://forge.softwareheritage.org/D1824 GNU Loader]'''
*** As part of the first step towards the implementation of Base Loader, GNU Loader was implemented.

===TO-DO:===
* Implement the Listers using the research done and the implementation plan made for Launchpad, Rubygem.
* Find the workarounds to solve the challenges in making the Maven and NuGET(.NET) Lister.
* Work on the remaining steps in order to complete the Base Package Manager Loader.

=== Learnings: ===
Working in Software Heritage was a wholesome experience. I got to learn a new thing almost every day. It would me injustice id I say I can account all my learnings in a section of a blog, however here are a list of few of most prominent once:
*Working on a huge codebase
*Plan and design before jumping to code
*Writing clean and well-commented code
*Difference between doing projects in college and in the industry(Spoiler Alert: '''A lot''')
*Multiple language integration in a python library (Used in CRAN Lister)
*Different programming methodologies explained to me by my mentors(eg [https://en.wikipedia.org/wiki/Test-driven_development TDD])
*Working on git and forge
*Docker
===Experience:===

=== Activity reports:===
* May 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00003.html Week 20 Second Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00010.html Week 21 Third Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00017.html Week 22 First Week (Coding)]
* June 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00009.html Week 23 Second Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00016.html Week 24 Third Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00026.html Week 25 Fourth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00033.html Week 26 Fifth Week (First Evaluation)]
* July 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00003.html Week 27 Sixth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00006.html Week 28 Seventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00011.html Week 29 Eight Week (Coding)(Work Summary)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00015.html Week 30 Nineth Week (Second Evaluation)]
* August 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00002.html Week 31 Tenth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00004.html Week 32 Eleventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 33 Twelfth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 34 Thirteenth Week (Final Evaluation)]

== Links ==
* [https://forge.softwareheritage.org/source/swh-lister/ Lister source code repository]
* [https://forge.softwareheritage.org/source/swh-loader-core/ Loader source code repository]
* see project [https://summerofcode.withgoogle.com/projects/#5658995887439872 on the GSoC portal]

[[Category: Google Summer of Code]]
[[Category: Google Summer of Code 2019]]

Google Summer of Code 2019/Increase archive coverage

2019-08-21T17:58:04Z

Archit agrawal:

* '''Title:''' Increase archive coverage
* '''Description:''' As Software Heritage works on archiving and sharing source code, one of the major tasks is to ingest the latest source code available in the database from time to time and from all the possible sources where you can fetch the source code using listers and ingest them using loaders. Listers are components that crawl the APIs of software forges (e.g., Bitbucket, Gitorious, Sourceforge, ...) and return a list of the software available in it whereas Loaders take a bundle of software (tarball, Git repository ...) and load it into Software Heritage, by adapting it so that it matches the archive data model. The goal of this project is to increase the archive coverage by making listers and loaders for different websites that which stores source code, so that Software Heritage can fetch as much source code as possible and store it in the database to preserve it for future generations.
* '''Student:''' Archit Agrawal
** [https://forge.softwareheritage.org/p/nahimilega/ Forge activity]
* '''Mentors:'''
** Nicolas Dandrimont
** Antoine R. Dumont
* '''Experience:'''
* '''What I did:'''
** '''Listers:'''
*** Completed and merged
**** [https://forge.softwareheritage.org/rDLSfedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6 Phabricator Lister]
**** [https://forge.softwareheritage.org/D1482 GNU Lister]
**** [https://forge.softwareheritage.org/rDLSa9a37a85bf9efac416cfdd152588bf01b7a063b2 CRAN Lister]
**** [https://forge.softwareheritage.org/D1584 Packagist Lister]
**** [https://forge.softwareheritage.org/D1610 CGit Lister]
*** Did research on the methods that could be used to make following listers and made an implementation plan for the same
**** [https://forge.softwareheritage.org/T1734 Launchpad Lister]
**** [https://forge.softwareheritage.org/T1777 Rubygem Lister]
**** [https://forge.softwareheritage.org/T1718 NuGET(.NET) Lister]
**** [https://forge.softwareheritage.org/T1724 Maven Lister]
*** [https://forge.softwareheritage.org/rDLS08ade29e6de0616a3964360454ab52b58c082b75 Add tests to PyPI Lister]
*** [https://forge.softwareheritage.org/rDLSf424f07c7e628eb7a19d25f4fdb749682d97a21f Refractor base tests for listers]
***[https://forge.softwareheritage.org/D1441 Add documentation on *How to run a new lister*]
** '''Loaders:'''
*** Base Loader
*** GNU Loader

* '''Learnings:'''
**How to work on a huge codebase
**How to plan and design before jumping to code
**Writing clean and well-commented code
**How are projects are made in the industry different from making projects in college(Spoiler Alert: '''A lot''')
**Multiple language integration in a python library (Used in CRAN Lister)
**Different programming methodologies explained to me by my mentors(eg [https://en.wikipedia.org/wiki/Test-driven_development TDD])
**Working on git and forge
**Docker

* '''TO-DO:'''
* '''Activity reports:'''
* May 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00003.html Week 20 Second Week(Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00010.html Week 21 Third Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00017.html Week 22 First Week (Coding)]
* June 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00009.html Week 23 Second Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00016.html Week 24 Third Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00026.html Week 25 Fourth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00033.html Week 26 Fifth Week (First Evaluation)]
* July 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00003.html Week 27 Sixth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00006.html Week 28 Seventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00011.html Week 29 Eight Week (Coding)(Work Summary)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00015.html Week 30 Nineth Week (Second Evaluation)]
* August 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00002.html Week 31 Tenth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00004.html Week 32 Eleventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 33 Twelfth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 34 Thirteenth Week (Final Evaluation)]

== Links ==

* [https://forge.softwareheritage.org/source/swh-lister/ Lister source code repository]
* [https://forge.softwareheritage.org/source/swh-loader-core/ Loader source code repository]
* see project [https://summerofcode.withgoogle.com/projects/#5658995887439872 on the GSoC portal]

[[Category: Google Summer of Code]]
[[Category: Google Summer of Code 2019]]

Google Summer of Code 2019/Increase archive coverage

2019-08-21T15:04:44Z

Archit agrawal:

* '''Title:''' Increase archive coverage
* '''Description:''' As Software Heritage works on archiving and sharing source code, one of the major tasks is to ingest the latest source code available in the database from time to time and from all the possible sources where you can fetch the source code using listers and ingest them using loaders. Listers are components that crawl the APIs of software forges (e.g., Bitbucket, Gitorious, Sourceforge, ...) and return a list of the software available in it whereas Loaders take a bundle of software (tarball, Git repository ...) and load it into Software Heritage, by adapting it so that it matches the archive data model. The goal of this project is to increase the archive coverage by making listers and loaders for different websites that which stores source code, so that Software Heritage can fetch as much source code as possible and store it in the database to preserve it for future generations.
* '''Student:''' Archit Agrawal
** [https://forge.softwareheritage.org/p/nahimilega/ Forge activity]
* '''Mentors:'''
** Nicolas Dandrimont
** Antoine R. Dumont
* '''Experience:'''
* '''What I did:'''
** '''Listers:'''
*** Completed and merged
**** [https://forge.softwareheritage.org/rDLSfedfd73c8e4be8ce1d08b31c9a5cb99f9ca40fd6 Phabricator Lister]
**** [https://forge.softwareheritage.org/D1482 GNU Lister]
**** [https://forge.softwareheritage.org/rDLSa9a37a85bf9efac416cfdd152588bf01b7a063b2 CRAN Lister]
**** [https://forge.softwareheritage.org/D1584 Packagist Lister]
**** [https://forge.softwareheritage.org/D1610 CGit Lister]
*** [https://forge.softwareheritage.org/rDLS08ade29e6de0616a3964360454ab52b58c082b75 Add tests to PyPI Lister]
*** [https://forge.softwareheritage.org/rDLSf424f07c7e628eb7a19d25f4fdb749682d97a21f Refractor base tests for listers]
***[https://forge.softwareheritage.org/D1441 Add documentation on *How to run a new lister*]
*** Did research on the methods that could be used to make following listers and made an implementation plan for the same:
**** [https://forge.softwareheritage.org/T1734 Launchpad Lister]
**** [https://forge.softwareheritage.org/T1777 Rubygem Lister]
**** [https://forge.softwareheritage.org/T1718 NuGET(.NET) Lister]
**** [https://forge.softwareheritage.org/T1724 Maven Lister]
** '''Loaders:'''
*** Base Loader
*** GNU Loader

*'''Challenges:'''
* '''Learnings:'''
* '''TO-DO:'''
* '''Activity reports:'''
* May 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00003.html Week 20 Second Week(Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00010.html Week 21 Third Week (Community Bonding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-05/msg00017.html Week 22 First Week (Coding)]
* June 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00009.html Week 23 Second Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00016.html Week 24 Third Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00026.html Week 25 Fourth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-06/msg00033.html Week 26 Fifth Week (First Evaluation)]
* July 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00003.html Week 27 Sixth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00006.html Week 28 Seventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00011.html Week 29 Eight Week (Coding)(Work Summary)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-07/msg00015.html Week 30 Nineth Week (Second Evaluation)]
* August 2019
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00002.html Week 31 Tenth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00004.html Week 32 Eleventh Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 33 Twelfth Week (Coding)]
** [https://sympa.inria.fr/sympa/arc/swh-devel/2019-08/msg00008.html Week 34 Thirteenth Week (Final Evaluation)]

== Links ==

* [https://forge.softwareheritage.org/source/swh-lister/ Lister source code repository]
* [https://forge.softwareheritage.org/source/swh-loader-core// Loader source code repository]
* see project [https://summerofcode.withgoogle.com/projects/#5658995887439872 on the GSoC portal]

[[Category: Google Summer of Code]]
[[Category: Google Summer of Code 2019]]

Google Summer of Code 2019/Increase archive coverage

2019-08-21T09:49:59Z

Archit agrawal:

Google Summer of Code 2019/Increase archive coverage

2019-08-18T16:38:57Z

Archit agrawal: /* General information */

Google Summer of Code 2019/Increase archive coverage

2019-07-19T20:08:54Z

Archit agrawal:

Google Summer of Code 2019/Increase archive coverage

2019-07-19T19:44:45Z

Archit agrawal:

Google Summer of Code 2019/Increase archive coverage

2019-07-19T19:44:10Z

Archit agrawal:

Google Summer of Code 2019/Increase archive coverage

2019-06-22T08:16:46Z

Archit agrawal:

Google Summer of Code 2019/Increase archive coverage

2019-06-14T13:24:26Z

Archit agrawal:

Google Summer of Code 2019/Increase archive coverage

2019-05-29T11:00:37Z

Archit agrawal:

Google Summer of Code 2019/Increase archive coverage

2019-05-29T11:00:09Z

Archit agrawal:

Suggestion box: source code to add

2019-05-29T10:53:53Z

Archit agrawal:

The [[Archive]] is growing organically. We started "small", tracking 3 '''software origins''' (GitHub + Debian + GNU), and we will be adding new origins bit by bit, depending on the urgency of archiving them and available development energies to integrate them into Software Heritage.

Using this page you can add suggestions of software origins that we aren't following yet, but we should. You can include information about who to contact for technical collaboration, the urgency of archival, and other useful information. To that end, just add a row to the table below. Here some information about the meaning of the various columns.

Entries are currently listed simply in order of addition to this page; we will add more structure when the list will start growing.

=== Legend ===

;Software origin
: any (public accessible) "place" on the Internet that host software in source code form. Please provide a title for it and hyperlink it to the relevant URL
;Type of origin
: information about the kind of hosting, e.g., whether it is a forge, a collection of repositories, an homepage publishing tarball, or a one shot source code repository. For all kind of repositories please specify which VCS system is in use (Git, SVN, CVS, etc.)
;Contact
: who to contact for technical collaboration on how to best archive source code hosted on the software origin. You can list yourself if you're the relevant person, or provide the most relevant contact point if you know it
;Conservation status
: information about how likely it is that the software origin will disappear; high likelihood will make it more urgent for us to archive software hosted there. We suggest to use the [https://en.wikipedia.org/wiki/Conservation_status species conservation status], i.e., one of: Critically endangered (CR), Endangered (EN), Vulnerable (VU), Near threatened (NT), Least concern (LC).
;How to mirror
: (pointers to) technical information on how to do a full mirror of ''all'' the source code available at the software origin, ideally one shot and in batch
;How to keep up
: (pointers to) technical information on how to incrementally retrieve new source code accumulated since the last visit; usually this should be based on some kind of incremental change feed or event API
;Notes
: anything else you think we should know about this software origin
;Forge task
: pointer to the task on our [[forge]] tracking the work needed to ingest the software origin

== Suggestions ==

* https://notabug.org/ (customized gogs)
* https://gitgud.io/ (GitLab, run by Sapphire, a user-supported open source company)
* https://octo.sh/ (GitLab)
* https://chaos.expert/explore (GitLab by Chaos Computer Club)
* https://gitlab.coko.foundation/public (GitLab)
* https://git.teknik.io/explore/repos (Gitea)
* https://gitlab.gnome.org/explore/groups (GNOME software)
* https://launchpad.net/ (used by Ubuntu and others)
* https://archive.codeplex.com/ (was Microsoft's free, open source project hosting)
* https://wiki.p2pfoundation.net/List_of_Community-Hosted_GitLab_Instances
* https://0xacab.org/explore (Gitlab)
* https://git.fosscommunity.in/explore/projects (GitLab by Free Software Community of India)

{| class="wikitable"
!Software origin
!Type of origin
!Contact
!Conservation status
!How to mirror
!How to keep up
!Notes
![https://forge.softwareheritage.org/ Forge] task
|-
|'''(sample entry)''' ''GitHubBub forge''
|''Git hosting''
|''John Doe <john@example.com>''
|''LC''
|''retrieve full repo list at /api/list, then git clone on each entry''
|''poll RSS feed at /api/updates?since=YYYY-MM-DD''
|''nothing special to add''
|''[https://forge.softwareheritage.org/T123456 T123456]''
|-
|[https://bitbucket.org/ Bitbucket]
|Git and hg/Mercurial hosting
|
|style="background-color: lightgreen"| LC
|
|
|
|[https://forge.softwareheritage.org/T561 T561]
|-
|[https://sourceforge.net/ SourceForge]
|CVS, SVN, Mercurial, Git
|
|style="background-color: orange"|VU
|
|
|
|
|-
|[https://wiki.debian.org/Derivatives/Census all Debian derivatives]
|Debian-based distros
|Paul Wise <pabs@debian.org>
|varying, depending on the distro
|see [[Suggestion_box:_source_code_to_add/Debian_derivatives|details]]
|see [[Suggestion_box:_source_code_to_add/Debian_derivatives|details]]
|
|
|-
|[https://www.gentoo.org/ Gentoo]
|
|Johannes Kellner <gentoo@johannes-kellner.eu>
|style="background-color: lightgreen"|LC
|
|
|
|
|-
|[http://pauillac.inria.fr/~huet/cea.html Gérard Huet's seminal work on 3D]
|Scanned source code
|Gérard Huet <gerard.huet@inria.fr>
|style="background-color: red"|EN
|retrieve listing images from the web pages
|N/A
|links are half broken, yquem should be replaced with pauillac everywhere it appears
|
|-
|[http://www.softwarepreservation.org/projects Software Preservation Project]
|Website with a collection of archives
|Paul McJones <paul@mcjones.org>
|style="background-color: lightgreen"|LC
|
|
|
|
|-
|[https://code.nasa.gov/ 253 NASA open source software projects]
|
|
|style="background-color: lightgreen"|LC
|
|
|
|
|-
|[http://smaky.ch/ Smaky], the swiss micro-computer series
|[http://infini.smaky.ch/sources.html Partial code dump]
|[mailto:arnaud@epsitec.ch Pierre Arnaud] (current CEO of Epsitec) and/or [mailto:jean-daniel.nicoud@epfl.ch Jean-Daniel Nicoud] (founder of the computer series]
|style="background-color: red"|EN
|Probably manually
|No new updates
|Some references to this history: [http://www.memoires-informatiques.org/ Fondation Mémoires Informatiques], [http://smaky.ch/ Smaky.ch] (in particular, [http://smaky.ch/theme.php?id=lami the short history]
|
|-
|[https://www.wikidata.org/wiki/Wikidata:WikiProject_Informatics/FLOSS#Conservation_status_2 wikidata endangered software]
|depends on the "source code repository" property
|Loic Dachary <loic@dachary.org>
|style="background-color: yellow"|The risk is higher than [https://www.wikidata.org/wiki/Property_talk:P141 LC]
|A script should obtain the "source code repository" property for the software and mirror it depending on the [https://www.wikidata.org/wiki/Wikidata:WikiProject_Informatics/FLOSS#source_code_repository protocol] qualifier. If the "source code repository" is "no value", the [https://www.wikidata.org/wiki/Wikidata:WikiProject_Informatics/Software#streaming_media_URL streaming media URL] of the "preferred" [https://www.wikidata.org/wiki/Wikidata:WikiProject_Informatics/Software#software_version_.28P348.29 software version] should be downloaded instead.
|Once a copy is secured by software heritage, a URL to the software heritage repository should be added to the "source code repository" property and the "conservation status" property should be removed, meaning it is "least concerned" by default. The software will no longer show in the list of endangered software.
|This is work in progress, part of the [https://www.wikidata.org/wiki/Wikidata:WikiProject_Informatics/FLOSS wikidata FLOSS project] and the scripts do not exist yet.
|
|-
|historical KDE repos
|CVS/SVN/Git
|KDE sysadmin team <sysadmin@kde.org>
|style="background-color: yellow"|NT
|See [[Suggestion box: source code to add/KDE|details]]
|See [[Suggestion box: source code to add/KDE|details]]
|
|
|-
|[https://java.net/projects Java.net] & [https://kenai.com/ Kenai.com]
|hg, git, svn
|communitymanager@java.net
|style="background-color: black; color: white"|CR
|
|
|[https://community.oracle.com/community/java/javanet-forge-sunset Shutting down on April 28, 2017]
|
|-
|[https://fedorahosted.org/ fedorahosted.org]
|git, svn, hg, bzr
|[https://lists.fedoraproject.org/admin/lists/infrastructure@lists.fedoraproject.org infrastructure@lists.fedoraproject.org]
|style="background-color: black; color: white"|CR
|
|
|[https://communityblog.fedoraproject.org/fedorahosted-sunset-2017-02-28/ Shutting down on Feb. 28, 2017]
|
|-
|[http://www.societe-informatique-de-france.fr/wp-content/uploads/2015/12/1024-no7-Baude.pdf Langage symbolique d'Enseignement (LSE)]
|archives
|Association Enseignement Public et Informatique (EPI) <bureau@epi.asso.fr>
|style="background-color: black; color: white"|CR
|
|
| + educational software (INRP-CNDP)
|
|-
|[http://www.netlib.org The Netlib collection of numerical software]
|structured website with links to archives
|
|style="background-color: lightgreen"|LC
|
|
|many of these libraries are mirrored in sources already collected in Software Heritage; there is sure value in the curation information.
|
|-
|[https://codebender.cc/ codebender]
|IOT and educationnal ressources
|
|style="background-color: black; color: white"|CR
|
|
|closing expected at Dec31st, 2016 upon this announce by founders https://codebender.cc/next-chapter
|
|-
|[https://git.oschina.net/ OS China]
|Chinese GitHub equivalent
|
|style="background-color: lightgreen"|LC
|
|
|interesting test case for all the unicode tooling in Software Heritage
|
|-
|Usenet source code archives
|NNTP
|
|style="background-color: yellow"|NT
|crawl relevant newsgroup archives (e.g., at Google Groups), retrieve archives (possibly chunked), ingest
|one shot might be enough?
|suggestion by John Gilmore
|
|-
|[https://www.x.org/releases/ X11/XOrg archives]
|http
|
|style="background-color: lightgreen"|LC
|
|one shot might be enough
|
||''[https://forge.softwareheritage.org/T1774 T1774]''
|-
|[https://www.curseforge.com/ Curse mods]
|Code distributed as versioned tarballs
|
|style="background-color: lightgreen"|LC
|The API should be sufficient, maybe some scraping will be required
|
|
|
|-
|Symbian source code
|HG
|carlo.daffara@nodeweaver.eu
|style="background-color: orange"|VU
|
|
|Cloned from Nokia's Symbian Mercurial repository, a few days before the closure of the repo and the change to a proprietary license. The mercurial HG files are the only copy publicly available; I have made a snapshot of the code and placed on sourceforge- but missing is the entire project history and commit log.
|
|-
|[https://gist.github.com/ GitHub's gists]
|git
|
|style="background-color: lightgreen"|LC
|No proper listing API, but there's [https://developer.github.com/v3/gists/#list-all-public-gists an endpoint] to get gists created after a given date
|use the same endpoint
|
|
|-
|[https://codeberg.org/ Codeberg]
|git
|contact@codeberg.org
|style="background-color: lightgreen"|LC
|[https://github.com/go-gitea Gitea] API
|[https://github.com/go-gitea Gitea] API
|Codeberg e.V. is a Non-Profit Collaboration Community for Free and Open Source Projects
|
|-
|[https://puszcza.gnu.org.ua/ Puszcza]
|cvs/git/hg/svn, VCS snapshot tarballs, tarballs
|https://puszcza.gnu.org.ua/contact.php
|style="background-color: lightgreen"|LC
|Savane instance, https://download.gnu.org.ua/ (also has ftp), http://git.gnu.org.ua/
|Savane instance
|
|
|-

[[Category:Archive]]
[[Category:Suggestions]]

Google Summer of Code 2019/Increase archive coverage

2019-05-24T04:02:53Z

Archit agrawal:

Google Summer of Code 2019/Increase archive coverage

2019-05-15T15:06:37Z

Archit agrawal:

Google Summer of Code 2019/Increase archive coverage

2019-05-15T15:03:18Z

Archit agrawal:

Google Summer of Code 2019/Increase archive coverage

2019-05-15T14:56:03Z

Archit agrawal:

* '''Title:''' Increase archive coverage
* '''Description:''' As Software Heritage works on archiving and sharing source code. One of the major tasks is to ingest the latest source code available in the database from time to time and from all the possible sources where you can fetch the source code using listers and ingest them using loaders. Listers are components that crawl the APIs of software forges (e.g., Bitbucket, Gitorious, Sourceforge, ...) and return a list of the software available in it whereas Loaders take a bundle of software (tarball, Git repository, Python package, ...) and load it into Software Heritage, by adapting it so that it matches the archive data model.

So the goal of this project is to make specialised softwares which are called Listers and Loaders
The goal of this to increase the archive coverage by making listers and loaders for different websites that which stores source code, so that Software Heritage can fetch as much source code as possible and store it in the database to preserve it for future generations
* '''Student:''' Archit Agrawal
** [https://forge.softwareheritage.org/p/nahimilega/ forge activity]
* '''Mentors:'''
** Nicolas Dandrimont
** Antoine R. Dumont
* '''Activity reports:'''
**

== Links ==

* [http://example.com source code repository]
* see project [http://example.com on the GSoC portal]

[[Category:Google Summer of Code]]
[[Category:Google Summer of Code 2019]]

Google Season of Docs 2019

2019-05-13T20:28:09Z

Archit agrawal: /* General information */

[[File:GSoDLogo.png|800px]]

== General information ==

This page is the central point of information for [[Software Heritage]] participation into the [https://developers.google.com/season-of-docs/ Google Season of Docs] program.

Google Season of Docs is a program where Google pays technical writers stipends to work on free software projects such as Software Heritage. Each writer works with mentors from the community to complete a documentation project.

== I want to participate as a technical writer ==

Great!, we are very glad for your interest in contributing to Software Heritage and we are looking forward to work together.

=== Prerequisites ===

The following prerequisites apply to Software Heritage GSoD projects:

* [http://www.sphinx-doc.org/ Sphinx] is our documentation system of choice, you should be familiar with it to apply. In particular, we generally use reStructuredText markup and (for API references) Python docstrings with the [https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html Napoleon style]
* [https://git-scm.com Git] is our version control system of choice, you should be familiar with it to apply
* additional prerequisites depend on the project you will work on; check project descriptions for details

=== Before you apply ===

Here are the steps you should follow before applying, to make sure you have a general idea of the current state of Software Heritage technical documentation:

* Learn about our project via our [https://www.softwareheritage.org/ main website] and the actual [https://archive.softwareheritage.org/ source code archive]
* Check out the index of our [https://www.softwareheritage.org/community/developers/ resources for developers]
* In particular make sure to have a look at:
** our [https://wiki.softwareheritage.org/ public wiki]
** our [https://docs.softwareheritage.org/devel/ documentation index]
* Technical setup:
** Create an account on our [https://forge.softwareheritage.org development forge]
** Familiarize yourself with our [[Code review in Phabricator|code review workflow]]
** Make a simple change to the documentation of any one of our [https://docs.softwareheritage.org/devel/ software components] and submit it as a [https://forge.softwareheritage.org/differential/ diff] for code review, following the above workflow. Feel free to submit any patch you think it might be useful.

=== What to include in your application ===

Make sure that your application includes the following information:

* Describe the '''specific project''' you want to work on. What do you want to achieve? Why is it important? Why is it useful for Software Heritage? The project might be one of the project ideas that we have prepared below, or something else entirely that you want to contribute to Software Heritage. Your pet peeve, surprise us!
* Detail your '''work plan''': a brief description of how you plan to go about your project, including a list of ''deliverables'' and a ''timeline'' of when do you expect them to be available.
* Include a reference to '''the diff''' you submitted before applying (see the "Before you apply" section above).

== Ideas list ==

Below you can find a list of project ideas that are good options for a
reasonably sized GSoD project. They are just suggestion though, don't feel
obliged to pick one of them if there is nothing that fits your taste and
abilities. Feel free to propose something else that you are excited about and
that contributes to improve the Software Heritage documentation: we will be
happy to consider it!

=== Reorganize developer documentation as tutorials/how-to/discussions/references ===

Our [https://docs.softwareheritage.org/devel/ developer documentation] is not particularly structured, making it hard to use it properly.
We would like to reorganize it following the [https://www.youtube.com/watch?v=t4vKPhjcMZg tutorials/how-to/discussions/references taxonomy], or something equally sensible.

This project will require discussing the intended use cases of this page, conceptually structure the future version of it, implement it, and (re)write suitable language to introduce the various parts.

=== Write a high-level technical overview of the project, data model, and data flow ===

We have described the data model and archival data flow of Software Heritage in [https://upsilon.cc/~zack/research/publications/ipres-2018-doi.pdf various] [https://upsilon.cc/~zack/research/publications/cacm-2018-software-heritage.pdf scientific] [https://upsilon.cc/~zack/research/publications/msr-2019-swh.pdf papers], but haven't really worked on a general, high-level, technical presentation of it that targets developers.
The current [https://docs.softwareheritage.org/devel/swh-model/data-model.html#data-model data model documentation] has been copy-pasted from scientific papers.
We want to revisit it, to make sure the language description is suitable for a more general technical public, possibly complementing it with more abstract data-structure descriptions and cross-references to the code implementing the model and the workflow in the actual implementation.

=== Design and document consistent writing conventions for Python docstrings ===

The docstrings of the [https://docs.softwareheritage.org/devel/#components various software components] in the Software Heritage stack are not very consistent.
They have been written by a number of different developers, with varying degrees of English proficiency.

We would like to document, as a set of writing guidelines, how to write them to the benefits of current and future developers.
The guidelines should include what to write in docstrings and whatnot, writing styles and guidance, examples of good and bad content for them, etc.

To the extent it is possible, this project will also explore automating checks for guidelines conformance, in the form of basic structural (e.g., are all function parameters documented? is the function name correct? etc.) and writing checks (e.g., spellchecking, grammar checking, etc.).

=== Revamp new (code) contributor tutorial ===

The current [https://docs.softwareheritage.org/devel/#getting-started getting started documentation] include two overlapping documents: one about how to ''run'' a local instance of the full Software Heritage stack, another about how to create a local setup for current and future project ''developers''.
We would like to review those documents for consistency and, more generally, revamp documentation that will help wannabe code contributors to quickly get started and submit patches.

As part of this project we might also want to revamp glossary, data model and other cross-cutting documents that help newcomers understand where-is-what in the code base and how to practically prepare and send us patches.

=== Restructure wiki landing page as main community entry point ===

Our [[Main_Page|public wiki landing page]] is, well, a mess.
It has grown inorganically by accumulating links to pages used more as working drafts than reasonably structured documents.
We would like to restructure it to cater for various use cases (users, developers, students, curators, scientists, etc.) in the attempt of making it the primary entry point into Software Heritage for all community members.

This project will require discussing the various uses cases, design a suitable page layout, implement it in the wiki, write the introductory parts for each path into the wiki.
A related task, if time permits, will be to reorganize the wiki [[Special:Categories|category taxonomy]] and document when/how to use the available categories in the future.

Working on this project will require familiarity with [https://www.mediawiki.org/wiki/MediaWiki Mediawiki] and its markup language.

== Contact ==

GSoD applicants are encouraged to get in touch with the Software Heritage community using the standard development communication channels, i.e.:

* the #swh-devel IRC channel on [https://freenode.net Freenode]
* the [https://sympa.inria.fr/sympa/info/swh-devel swh-devel mailing list]

See our [https://www.softwareheritage.org/community/developers/ development information page] for more details.

== Timeline ==

See the official [https://developers.google.com/season-of-docs/docs/timeline Google Season of Docs timeline].

Google Season of Docs 2019

2019-05-13T20:22:24Z

Archit agrawal: /* General information */

[[File:GSoDLogo.png|800px]]

== General information ==

This page is the central point of information for [[Software Heritage]] participation into the [https://developers.google.com/season-of-docs/ Google Season of Docs] program.

Google Summer of Docs is a program where Google pays technical writers stipends to work on free software projects such as Software Heritage. Each writer works with mentors from the community to complete a documentation project.

== I want to participate as a technical writer ==

Great!, we are very glad for your interest in contributing to Software Heritage and we are looking forward to work together.

=== Prerequisites ===

The following prerequisites apply to Software Heritage GSoD projects:

* [http://www.sphinx-doc.org/ Sphinx] is our documentation system of choice, you should be familiar with it to apply. In particular, we generally use reStructuredText markup and (for API references) Python docstrings with the [https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html Napoleon style]
* [https://git-scm.com Git] is our version control system of choice, you should be familiar with it to apply
* additional prerequisites depend on the project you will work on; check project descriptions for details

=== Before you apply ===

Here are the steps you should follow before applying, to make sure you have a general idea of the current state of Software Heritage technical documentation:

* Learn about our project via our [https://www.softwareheritage.org/ main website] and the actual [https://archive.softwareheritage.org/ source code archive]
* Check out the index of our [https://www.softwareheritage.org/community/developers/ resources for developers]
* In particular make sure to have a look at:
** our [https://wiki.softwareheritage.org/ public wiki]
** our [https://docs.softwareheritage.org/devel/ documentation index]
* Technical setup:
** Create an account on our [https://forge.softwareheritage.org development forge]
** Familiarize yourself with our [[Code review in Phabricator|code review workflow]]
** Make a simple change to the documentation of any one of our [https://docs.softwareheritage.org/devel/ software components] and submit it as a [https://forge.softwareheritage.org/differential/ diff] for code review, following the above workflow. Feel free to submit any patch you think it might be useful.

=== What to include in your application ===

Make sure that your application includes the following information:

* Describe the '''specific project''' you want to work on. What do you want to achieve? Why is it important? Why is it useful for Software Heritage? The project might be one of the project ideas that we have prepared below, or something else entirely that you want to contribute to Software Heritage. Your pet peeve, surprise us!
* Detail your '''work plan''': a brief description of how you plan to go about your project, including a list of ''deliverables'' and a ''timeline'' of when do you expect them to be available.
* Include a reference to '''the diff''' you submitted before applying (see the "Before you apply" section above).

== Ideas list ==

Below you can find a list of project ideas that are good options for a
reasonably sized GSoD project. They are just suggestion though, don't feel
obliged to pick one of them if there is nothing that fits your taste and
abilities. Feel free to propose something else that you are excited about and
that contributes to improve the Software Heritage documentation: we will be
happy to consider it!

=== Reorganize developer documentation as tutorials/how-to/discussions/references ===

Our [https://docs.softwareheritage.org/devel/ developer documentation] is not particularly structured, making it hard to use it properly.
We would like to reorganize it following the [https://www.youtube.com/watch?v=t4vKPhjcMZg tutorials/how-to/discussions/references taxonomy], or something equally sensible.

This project will require discussing the intended use cases of this page, conceptually structure the future version of it, implement it, and (re)write suitable language to introduce the various parts.

=== Write a high-level technical overview of the project, data model, and data flow ===

We have described the data model and archival data flow of Software Heritage in [https://upsilon.cc/~zack/research/publications/ipres-2018-doi.pdf various] [https://upsilon.cc/~zack/research/publications/cacm-2018-software-heritage.pdf scientific] [https://upsilon.cc/~zack/research/publications/msr-2019-swh.pdf papers], but haven't really worked on a general, high-level, technical presentation of it that targets developers.
The current [https://docs.softwareheritage.org/devel/swh-model/data-model.html#data-model data model documentation] has been copy-pasted from scientific papers.
We want to revisit it, to make sure the language description is suitable for a more general technical public, possibly complementing it with more abstract data-structure descriptions and cross-references to the code implementing the model and the workflow in the actual implementation.

=== Design and document consistent writing conventions for Python docstrings ===

The docstrings of the [https://docs.softwareheritage.org/devel/#components various software components] in the Software Heritage stack are not very consistent.
They have been written by a number of different developers, with varying degrees of English proficiency.

We would like to document, as a set of writing guidelines, how to write them to the benefits of current and future developers.
The guidelines should include what to write in docstrings and whatnot, writing styles and guidance, examples of good and bad content for them, etc.

To the extent it is possible, this project will also explore automating checks for guidelines conformance, in the form of basic structural (e.g., are all function parameters documented? is the function name correct? etc.) and writing checks (e.g., spellchecking, grammar checking, etc.).

=== Revamp new (code) contributor tutorial ===

The current [https://docs.softwareheritage.org/devel/#getting-started getting started documentation] include two overlapping documents: one about how to ''run'' a local instance of the full Software Heritage stack, another about how to create a local setup for current and future project ''developers''.
We would like to review those documents for consistency and, more generally, revamp documentation that will help wannabe code contributors to quickly get started and submit patches.

As part of this project we might also want to revamp glossary, data model and other cross-cutting documents that help newcomers understand where-is-what in the code base and how to practically prepare and send us patches.

=== Restructure wiki landing page as main community entry point ===

Our [[Main_Page|public wiki landing page]] is, well, a mess.
It has grown inorganically by accumulating links to pages used more as working drafts than reasonably structured documents.
We would like to restructure it to cater for various use cases (users, developers, students, curators, scientists, etc.) in the attempt of making it the primary entry point into Software Heritage for all community members.

This project will require discussing the various uses cases, design a suitable page layout, implement it in the wiki, write the introductory parts for each path into the wiki.
A related task, if time permits, will be to reorganize the wiki [[Special:Categories|category taxonomy]] and document when/how to use the available categories in the future.

Working on this project will require familiarity with [https://www.mediawiki.org/wiki/MediaWiki Mediawiki] and its markup language.

== Contact ==

GSoD applicants are encouraged to get in touch with the Software Heritage community using the standard development communication channels, i.e.:

* the #swh-devel IRC channel on [https://freenode.net Freenode]
* the [https://sympa.inria.fr/sympa/info/swh-devel swh-devel mailing list]

See our [https://www.softwareheritage.org/community/developers/ development information page] for more details.

== Timeline ==

See the official [https://developers.google.com/season-of-docs/docs/timeline Google Season of Docs timeline].