Software Heritage Wiki - User contributions [en]

Google Summer of Code 2019

2019-05-09T15:39:20Z

AntoineLambert:

[[File:GSoCLogo.png|1024px]]

== General information ==

This page is the central point of information for [[Software Heritage]] participation into the [https://summerofcode.withgoogle.com/ Google Summer of Code] program.

Google Summer of Code is a program where Google pays students stipends to work over the (northern hemisphere) summer on free software projects such as Software Heritage. Each student works with mentors from the community to complete a software project.

== Accepted projects ==

* [[Google Summer of Code 2019/Graph compression|Graph compression on the development history of software]] - Thibault Allançon
* [[Google Summer of Code 2019/Web UI improvements|Web UI improvements]] - Kalpit Kothari
* [[Google Summer of Code 2019/Increase archive coverage|Increase archive coverage]] - Archit Agrawal

== I want to participate as a student ==

Great!, we are very glad for your interest in contributing to Software Heritage and we are looking forward to work together.

=== Prerequisites ===

The following prerequisites apply to Software Heritage GSoC projects:

* [https://www.python.org Python] 3 is our language of choice, you should be fluent with that language to apply
* [https://git-scm.com Git] is our version control system of choice, you should be familiar with it to apply
* additional prerequisites depend on the project you will work on; check project descriptions for details

=== Before you apply ===

Here are the steps you should follow before applying, to make sure you have a good grasp of what we are doing at Software Heritage and how we do it:

# Follow our [https://docs.softwareheritage.org/devel/getting-started.html getting started guide]: it will make sure you can locally run a (small) copy of the archive and ingest source code into it
# Create an account on our [https://forge.softwareheritage.org development forge]
# Familiarize yourself with our [[Code review in Phabricator|code review workflow]]
# Make a simple change to any one of our [https://docs.softwareheritage.org/devel/ software components] and submit it as a [https://forge.softwareheritage.org/differential/ diff] for code review, following the above workflow. [[Easy hacks]] and [https://forge.softwareheritage.org/project/view/20/ Web UI] issues are good options for what to fix, but feel free to submit any patch you think it might be useful.

=== What to include in your application ===

Make sure that your application includes the following information:

* Describe the '''specific project''' you want to work on. What do you want to achieve? Why is it important? Why is it useful for Software Heritage? The project might be one of the project ideas that we have prepared below, or something else entirely that you want to contribute to Software Heritage. Your source code archival pet peeve, surprise us!
* Detail your '''work plan''': a brief description of how you plan to go about your project, including a list of ''deliverables'' and a ''timeline'' of when do you expect them to be available.
* Include a reference to '''the diff''' you submitted before applying (see the "Before you apply" section above).

== Ideas list ==

Below you can find a list of project ideas that are good options for a
reasonably sized GSoC project. They are just suggestion though, don't feel
obliged to pick one of them if there is nothing that fits your taste and
abilities. Feel free to propose something else that you are excited about and
that contributes to improve the Software Heritage archive: we will be happy to
consider it!

=== Increase archive coverage ===

Software Heritage aims to archive ''all publicly available software source
code''. We naturally started with the place where most of the software is
easily available today: git repositories on GitHub.

As Software Heritage grows, we're incrementally increasing archive coverage by
expanding the sources from which we archive software; a list of currently
crawled sources is listed on the
[https://archive.softwareheritage.org main archive page]. As you can see there,
we have already built ways of archiving Mercurial repositories, Debian packages,
PyPI bundles, and more.

Further expansions of archive coverage are very suitable GSoC project.

Practically, to expand archive coverage two kinds of software components need
to be implemented: listers and loaders.

'''Listers''' are components that crawl the APIs of software
[https://en.wikipedia.org/wiki/Forge_(software) forges] (e.g., Bitbucket,
Gitorious, Sourceforge, ...) or package managers (a larges list is maintained
by [https://libraries.io Libraries.io]) and return a list of the software
available in it. See the official
[https://docs.softwareheritage.org/devel/swh-lister/ listers documentation] for
more details.

'''Loaders''' take a bundle of software (tarball, Git repository, Python
package, ...) and load it into Software Heritage, by adapting it so that it
matches the archive
[https://docs.softwareheritage.org/devel/swh-model/data-model.html data model].

While listers answer questions like "what are all the repositories available on
npm.org?", a loader addresses the "how do I load the NPM package I downloaded
into Software Heritage?" problem.

Writing a missing lister or a loader is a great way to contribute to expand the
coverage of the Software Heritage archive! Feel free to propose the
implementation of one (or several!) listers or loaders that are currently
missing. For inspiration you can check out our [[Suggestion box]] for code to
archive, or propose your favorite missing forge or package repository.

=== Mine information from archived content ===

In addition to archival, Software Heritage indexes the retrieved source code
artifacts, to enable semantic searches on the archive and scientific research.

Indexing can happen at the individual file-level (e.g., detect the programming
language the file is written in or the license declared in its header), or at
more coarse grained granularity (e.g., what metadata are declared for the most
recently archived version of a given project).

A number of indexes are [https://forge.softwareheritage.org/source/swh-indexer/ currently supported],
such as:

* file level mining:
** MIME type detection (using libmagic)
** license detection (using FOSSology/nomossa)
** language detection (using Pygments)
** ctags extraction (using universal-ctags)
* project level mining:
** Ruby gemspec metadata
** Python PKG-INFO metadata
** Maven pom.xml metadata
** NPM package.json metadata

Writing additional indexers that extract more information from archived source
code is welcome and would constitute a suitable GSoC project.

Name the kind of data mining you want to do!

For inspiration you can have a look at [https://libraries.io Libraries.io], as
most package formats/package managers support dedicated ways of expressing
metadata and we only support a small number of them up-to-now. But do not
restrict your ambition to those, any kind of data extraction/mining you want to
do on the archive could work.

=== Improve and extend the archive Web UI ===

As you probably know already, The Software Heritage archive can be
[https://archive.softwareheritage.org browsed on the Web]. The
[https://forge.softwareheritage.org/source/swh-web/ code] powering that
interface is a Django application that also implements a
[https://archive.softwareheritage.org/api/ Web API].

Several improvements are possible on the archive Web interface and would make
great GSoC projects, some ideas to whet your appetite:

* improve navigation on mobile devices and browsers
* add new source code search criteria and improve the search interface
* add developer-oriented features, e.g., source file history, blame/praise interface, in-browser edit (with patch download), ...
* improve [https://www.w3.org/WAI/ accessibility]
* add missing API endpoints (name your pet peeves!)
* add end to end tests using [https://www.seleniumhq.org/ Selenium]

=== Research internships ===

For the more research-inclined students, we also maintain a separate list of [[Internships]].
Any topic there is also a viable GSoC project.

== Contact ==

GSoC students are encouraged to get in touch with the Software Heritage community using the standard development communication channels, i.e.:

* the #swh-devel IRC channel on [https://freenode.net Freenode]
* the [https://sympa.inria.fr/sympa/info/swh-devel swh-devel mailing list]

See our [https://www.softwareheritage.org/community/developers/ development information page] for more details.

== Timeline ==

See the official [https://developers.google.com/open-source/gsoc/timeline Google Summer of Code timeline].

[[Category:Google Summer of Code]]
[[Category:Google Summer of Code 2019]]

Google Summer of Code 2019

2019-03-07T16:30:03Z

AntoineLambert:

[[File:GSoCLogo.png|1024px]]

== General information ==

This page is the central point of information for [[Software Heritage]] participation into the [https://summerofcode.withgoogle.com/ Google Summer of Code] program.

Google Summer of Code is a program where Google pays students stipends to work over the (northern hemisphere) summer on free software projects such as Software Heritage. Each student works with mentors from the community to complete a software project.

== I want to participate as a student ==

Great!, we are very glad for your interest in contributing to Software Heritage and we are looking forward to work together.

=== Prerequisites ===

The following prerequisites apply to Software Heritage GSoC projects:

* [https://www.python.org Python] 3 is our language of choice, you should be fluent with that language to apply
* [https://git-scm.com Git] is our version control system of choice, you should be familiar with it to apply
* additional prerequisites depend on the project you will work on; check project descriptions for details

=== Before you apply ===

Here are the steps you should follow before applying, to make sure you have a good grasp of what we are doing at Software Heritage and how we do it:

# Follow our [https://docs.softwareheritage.org/devel/getting-started.html getting started guide]: it will make sure you can locally run a (small) copy of the archive and ingest source code into it
# Create an account on our [https://forge.softwareheritage.org development forge]
# Familiarize yourself with our [[Code review in Phabricator|code review workflow]]
# Make a simple change to any one of our [https://docs.softwareheritage.org/devel/ software components] and submit it as a [https://forge.softwareheritage.org/differential/ diff] for code review, following the above workflow. [[Easy hacks]] and [https://forge.softwareheritage.org/project/view/20/ Web UI] issues are good options for what to fix, but feel free to submit any patch you think it might be useful.

=== What to include in your application ===

Make sure that your application includes the following information:

* Describe the '''specific project''' you want to work on. What do you want to achieve? Why is it important? Why is it useful for Software Heritage? The project might be one of the project ideas that we have prepared below, or something else entirely that you want to contribute to Software Heritage. Your source code archival pet peeve, surprise us!
* Detail your '''work plan''': a brief description of how you plan to go about your project, including a list of ''deliverables'' and a ''timeline'' of when do you expect them to be available.
* Include a reference to '''the diff''' you submitted before applying (see the "Before you apply" section above).

== Ideas list ==

Below you can find a list of project ideas that are good options for a
reasonably sized GSoC project. They are just suggestion though, don't feel
obliged to pick one of them if there is nothing that fits your taste and
abilities. Feel free to propose something else that you are excited about and
that contributes to improve the Software Heritage archive: we will be happy to
consider it!

=== Increase archive coverage ===

Software Heritage aims to archive ''all publicly available software source
code''. We naturally started with the place where most of the software is
easily available today: git repositories on GitHub.

As Software Heritage grows, we're incrementally increasing archive coverage by
expanding the sources from which we archive software; a list of currently
crawled sources is listed on the
[https://archive.softwareheritage.org main archive page]. As you can see there,
we have already built ways of archiving Mercurial repositories, Debian packages,
PyPI bundles, and more.

Further expansions of archive coverage are very suitable GSoC project.

Practically, to expand archive coverage two kinds of software components need
to be implemented: listers and loaders.

'''Listers''' are components that crawl the APIs of software
[https://en.wikipedia.org/wiki/Forge_(software) forges] (e.g., Bitbucket,
Gitorious, Sourceforge, ...) or package managers (a larges list is maintained
by [https://libraries.io Libraries.io]) and return a list of the software
available in it. See the official
[https://docs.softwareheritage.org/devel/swh-lister/ listers documentation] for
more details.

'''Loaders''' take a bundle of software (tarball, Git repository, Python
package, ...) and load it into Software Heritage, by adapting it so that it
matches the archive
[https://docs.softwareheritage.org/devel/swh-model/data-model.html data model].

While listers answer questions like "what are all the repositories available on
npm.org?", a loader addresses the "how do I load the NPM package I downloaded
into Software Heritage?" problem.

Writing a missing lister or a loader is a great way to contribute to expand the
coverage of the Software Heritage archive! Feel free to propose the
implementation of one (or several!) listers or loaders that are currently
missing. For inspiration you can check out our [[Suggestion box]] for code to
archive, or propose your favorite missing forge or package repository.

=== Mine information from archived content ===

In addition to archival, Software Heritage indexes the retrieved source code
artifacts, to enable semantic searches on the archive and scientific research.

Indexing can happen at the individual file-level (e.g., detect the programming
language the file is written in or the license declared in its header), or at
more coarse grained granularity (e.g., what metadata are declared for the most
recently archived version of a given project).

A number of indexes are [https://forge.softwareheritage.org/source/swh-indexer/ currently supported],
such as:

* file level mining:
** MIME type detection (using libmagic)
** license detection (using FOSSology/nomossa)
** language detection (using Pygments)
** ctags extraction (using universal-ctags)
* project level mining:
** Ruby gemspec metadata
** Python PKG-INFO metadata
** Maven pom.xml metadata
** NPM package.json metadata

Writing additional indexers that extract more information from archived source
code is welcome and would constitute a suitable GSoC project.

Name the kind of data mining you want to do!

For inspiration you can have a look at [https://libraries.io Libraries.io], as
most package formats/package managers support dedicated ways of expressing
metadata and we only support a small number of them up-to-now. But do not
restrict your ambition to those, any kind of data extraction/mining you want to
do on the archive could work.

=== Improve and extend the archive Web UI ===

As you probably know already, The Software Heritage archive can be
[https://archive.softwareheritage.org browsed on the Web]. The
[https://forge.softwareheritage.org/source/swh-web/ code] powering that
interface is a Django application that also implements a
[https://archive.softwareheritage.org/api/ Web API].

Several improvements are possible on the archive Web interface and would make
great GSoC projects, some ideas to whet your appetite:

* improve navigation on mobile devices and browsers
* add new source code search criteria and improve the search interface
* add developer-oriented features, e.g., source file history, blame/praise interface, in-browser edit (with patch download), ...
* improve [https://www.w3.org/WAI/ accessibility]
* add missing API endpoints (name your pet peeves!)
* add end to end tests using [https://www.seleniumhq.org/ Selenium]

=== Research internships ===

For the more research-inclined students, we also maintain a separate list of [[Internships]].
Any topic there is also a viable GSoC project.

== Contact ==

GSoC students are encouraged to get in touch with the Software Heritage community using the standard development communication channels, i.e.:

* the #swh-devel IRC channel on [https://freenode.net Freenode]
* the [https://sympa.inria.fr/sympa/info/swh-devel swh-devel mailing list]

See our [https://www.softwareheritage.org/community/developers/ development information page] for more details.

== Timeline ==

See the official [https://developers.google.com/open-source/gsoc/timeline Google Summer of Code timeline].

Code review in Phabricator

2019-02-10T17:05:29Z

AntoineLambert:

We use the [[Differential]] application of [[Phabricator]] to perform [[code review|code reviews]] in the context of [[Software Heritage]].

* we use Git and history.immutable=true (but beware as that is partly a Phabricator misnomer, read on)
* when code reviews are required, developers will be allowed to push directly to master once an accepted Differential diff exists

== Configuration ==

=== Arcanist configuration ===

When using git, [[Arcanist]] by default mess with the local history, rewriting commits at the time of first submission.<br />
To avoid that we use so called [https://secure.phabricator.com/book/phabricator/article/arcanist_new_project/#history-mutability-git history immutability].

To that end, you shall configure your <tt>arc</tt> accordingly:
<pre>
arc set-config history.immutable true
</pre>

Note that this does ''not'' mean that you are forbidden to rewrite your local branches (e.g., with <tt>git rebase</tt>).
Quite the contrary: you are encouraged to locally rewrite branches before pushing to ensure that commits are logically separated and your commit history easy to bisect.
The above setting just means that ''arc'' will not rewrite commit history under your nose.

=== Enabling <tt>git push</tt> to our forge ===

The way we've configured our review setup for continuous integration needs you to configure git to allow pushes to our forge. There's two ways you can do this : setting a ssh key to push over ssh, or setting a specific password for git pushes over https.

==== SSH key for pushes ====

In your forge User settings page (On the top right, click on your avatar, then click ''Settings''), you have access to a ''Authentication'' > ''SSH Public Keys'' section (Direct link: <tt>hxxps://forge.softwareheritage.org/settings/user/'''<your username>'''/page/ssh/</tt>). You then have the option to upload a SSH public key, which will authenticate your pushes.

You then need to configure ssh/git to use that key pair, for instance by editing the <tt>~/.ssh/config</tt> file.

Finally, you should configure git to push over ssh when pushing to https://forge.softwareheritage.org, by running the following command:
git config --global url.git@forge.softwareheritage.org:.pushInsteadOf https://forge.softwareheritage.org

This lets git know that it should use <tt>git@forge.softwareheritage.org:</tt> as a base url when pushing repositories cloned from forge.softwareheritage.org over https.

==== VCS password for pushes ====

If you're not comfortable setting up SSH to upload your changes, you have the option of setting a VCS password. This password, ''separate from your account password'', allows Phabricator to authenticate your uploads over HTTPS.

In your forge User settings page (On the top right, click on your avatar, then click ''Settings''), you need to use the ''Authentication'' > ''VCS Password'' section to set your VCS password (Direct link: <tt>hxxps://forge.softwareheritage.org/settings/user/'''<your username>'''/page/vcspassword/</tt>).

== Workflow ==

* work in a feature branch: <tt>git checkout -b my-feat</tt>
* initial review request: hack/commit/hack/commit ; <tt>arc diff origin/master</tt>
* react to change requests: hack/commit/hack/commit ; <tt>arc diff --update Dxx origin/master</tt>
* landing change: <tt>git checkout master ; git merge my-feat ; git push</tt>

=== Starting a new feature and submit it for review ===

Use a '''one branch per feature''' workflow, with well-separated ''logical commits'':
<pre>
git checkout -b my-shiny-feature
... hack hack hack ...
git commit -m 'architecture skeleton for my-shiny-feature'
... hack hack hack ...
git commit -m 'my-shiny-feature: implement module foo'
... etc ...
</pre>

To '''submit your code for review''' the first time:
<pre>
arc diff origin/master
</pre>
arc will prompt for a '''code review message'''. Provide the following information:
* first line: ''short description'' of the overall work (i.e., the feature you're working on). This will become the title of the review
* ''Summary'' field (optional): ''long description'' of the overall work; the field can continue in subsequent lines, up to the next field. This will become the "Summary" section of the review
* ''Test Plan'' field (optional): write here if something special is needed to test your change
* ''Reviewers'' field (optional): the (Phabricator) name(s) of desired reviewers. If you don't specify one (recommended) the default reviewers will be chosen
* ''Subscribers'' field (optional): the (Phabricator) name(s) of people that will be notified about changes to this review request. In most cases it should be left empty

For example:
<pre>
mercurial loader

Summary: first stab at a mercurial loader (T329)

The implementation follows the plan detailed in F2F discussion with @foo.

Performances seem decent enough for a first trial (XXX seconds for YYY repository
that contains ZZZ patches).

Test plan:

Reviewers:

Subscribers: foo
</pre>

After completing the message arc will submit the review request and tell you its number and URL:
<pre>
[...]
Created a new Differential revision:
Revision URI: https://forge.softwareheritage.org/Dxx
</pre>

=== Updating your branch to reflect requested changes ===

Your feature might get accepted as is, YAY!
Or, reviewers might request changes; no big deal!

Use the Differential web UI to follow-up to received comments, if needed.

To implement requested changes in the code, hack on your branch as usual by:

* adding new commits, and/or
* rewriting old commits with git rebase (to preserve a nice, easy to bisect history)

When you're ready to '''update your review request''':
<pre>
arc diff --update Dxx origin/master
</pre>

Arc will prompt you for a message: describe what you've changed w.r.t. the previous review request, free form.
Your message will become the changelog entry in Differential for this new version of the diff.

Differential only care about the code diff, and not about the commits or their order.
Therefore each "update" can be a completely different series of commits, possibly rewritten from the previous submission.

=== Landing your change onto master ===

Once your change has been approved in Differential, you will be able to land it onto the master branch.

Before doing so, you're encouraged to '''clean up your git commit history''', reordering/splitting/merging commits as needed to have separate logical commits and an easy to bisect history.
The correspondence between the accepted review and pushed code is checked by looking only at the code diff, so commit fiddling will not impact your ability to push to master.

Once you're happy you can '''push to origin/master''' directly, e.g.:
<pre>
git checkout master
git merge my-shiny-feature
git push
</pre>

Optionally you can then delete your local feature branch:
<pre>
git branch -d my-shiny-feature
</pre>

=== Reviewing locally / landing someone else's changes ===

You can do local reviews of code with arc patch:

<pre>
arc patch Dxyz
</pre>

This will create a branch '''arcpatch-Dxyz''' containing the changes on your local checkout.

You can then merge those changes upstream with

<pre>
git checkout master
git merge --ff arcpatch-Dxyz
git push origin master
</pre>

== See also ==

* [[Code review]] for guidelines on how code is reviewed when developing for Software Heritage

[[Category:Software development]]

Google Summer of Code 2019

2019-02-05T15:42:25Z

AntoineLambert:

[[File:GSoCLogo.png|1024px]]

== General information ==

This page is the central point of information for [[Software Heritage]] participation into the [https://summerofcode.withgoogle.com/ Google Summer of Code] program.

Google Summer of Code is a program where Google pays students stipends to work over the (northern hemisphere) summer on free software projects such as Software Heritage. Each student works with mentors from the community to complete a software project.

== I want to participate as a student ==

Great!, we are very glad for your interest in contributing to Software Heritage and we are looking forward to work together.

=== Prerequisites ===

The following prerequisites apply to Software Heritage GSoC projects:

* [https://www.python.org Python] 3 is our language of choice, you should be fluent with that language to apply
* [https://git-scm.com Git] is our version control system of choice, you should be familiar with it to apply
* additional prerequisites depend on the project you will work on; check project descriptions for details

=== Before you apply ===

Here are the steps you should follow before applying, to make sure you have a good grasp of what we are doing at Software Heritage and how we do it:

# Follow our [https://docs.softwareheritage.org/devel/getting-started.html getting started guide]: it will make sure you can locally run a (small) copy of the archive and ingest source code into it
# Create an account our [https://forge.softwareheritage.org development forge]
# Familiarize yourself with our [[Code review in Phabricator|code review workflow]]
# Make a simple change to any one of our [https://docs.softwareheritage.org/devel/ software components] and submit it as a [https://forge.softwareheritage.org/differential/ diff] for code review, following the above workflow. [[Easy hacks]] and [https://forge.softwareheritage.org/project/view/20/ Web UI] issues are good options for what to fix, but feel free to submit any patch you think it might be useful.

=== What to include in your application ===

Make sure that your application includes the following information:

* Describe the '''specific project''' you want to work on. What do you want to achieve? Why is it important? Why is it useful for Software Heritage? The project might be one of the project ideas that we have prepared, or something else entirely that you want to contribute to Software Heritage. Your source code archival pet peeve, surprise us!
* Detail your '''work plan''': a brief description of how you plan to go about your project, including a list of ''deliverables'' and a ''timeline'' of when do you expect them to be available.
* Include a reference to '''the diff''' you submitted before applying (see the "Before you apply" section above).

== Ideas list ==

Below you can find a list of project ideas that are good options for a reasonably sized GSoC project.
They are just suggestion though, don't feel obliged to pick one of them if there is nothing that fits your taste and abilities.
Feel free to propose something else that you are excited about and that contributes to improve the Software Heritage archive: we will be happy to consider it!

=== Increase archive coverage ===

Software Heritage aims to archive '''all the software'''. We naturally started with the place where most of the software is easily available today: git repositories on GitHub.

As Software Heritage grows, we're incrementally trying to increase the coverage of the archive by expanding the sources from which we archive software. We built ways of archiving things like mercurial repositories, Debian packages, pypi bundles, etc.

Expanding the coverage of the archive has two different components:

1. Create '''origin listers'''. Listers are pieces of code that crawl the APIs of Software Forges[https://en.wikipedia.org/wiki/Forge_(software)] (like Bitbucket, Gitorious, Sourceforge, NPM...) and return a list of the software available in it. The documentation on listers is here: https://docs.softwareheritage.org/devel/swh-lister/index.html

2. Create '''loaders'''. Loaders take a bundle of software (tarball, git repository, Python package, ...) and load it into Software Heritage, by adapting it so that it matches our uniform data model[https://docs.softwareheritage.org/devel/swh-model/data-model.html].

In a few words, a lister can be a way of asking "what are all the repositories available on npm.org?", while a loader would be "how do I load the NPM package I downloaded into Software Heritage?".

Writing a lister or a loader is a great way to contribute to Software Heritage by expanding its coverage! We have a list of software sources we would like to archive here[https://wiki.softwareheritage.org/wiki/Suggestion_box:_source_code_to_add], but you're free to suggest more.

=== Mine information from archived content ===

'''TODO'''

=== Improve and extend the archive Web UI ===

In order to easily navigate into the archive content, a [https://forge.softwareheritage.org/source/swh-web/ web application] is currently developed.
So far it offers the following main features:
* programmatic access to the content of the archive via the [https://archive.softwareheritage.org/api/ Software Heritage API]
* in-browser navigation of the content of the archive via the [https://archive.softwareheritage.org/browse/ Software Heritage browse UI]

There are still numerous improvements and new features to add to that web application, for instance:
* add new API endpoints
* improve overall design
* improve navigation for mobile browsers
* add new source code search criteria and improve the search interface
* implement new developer oriented features: source file history, blame interface, ...
* improve web application [https://www.w3.org/WAI/ accessibility]

If you are interested in web development and want to contribute to the application
enabling users to navigate in the biggest public source code archive collected so far,
feel free to apply.

== Contact ==

GSoC students are encouraged to get in touch with the Software Heritage community using the standard development communication channels, i.e.:

* the #swh-devel IRC channel on [https://freenode.net Freenode]
* the [https://sympa.inria.fr/sympa/info/swh-devel swh-devel mailing list]

See our [https://www.softwareheritage.org/community/developers/ development information page] for more details.

== Timeline ==

See the official [https://developers.google.com/open-source/gsoc/timeline Google Summer of Code timeline].

Google Summer of Code 2019

2019-02-05T13:28:52Z

AntoineLambert:

[[File:GSoCLogo.png|1024px]]

== General information ==

This page is the central point of information for [[Software Heritage]] participation into the [https://summerofcode.withgoogle.com/ Google Summer of Code] program.

Google Summer of Code is a program where Google pays students stipends to work over the (northern hemisphere) summer on free software projects such as Software Heritage. Each student works with mentors from the community to complete a software project.

== I want to participate as a student ==

Great!, we are very glad for your interest in contributing to Software Heritage and we are looking forward to work together.

=== Prerequisites ===

The following prerequisites apply to Software Heritage GSoC projects:

* [https://www.python.org Python] 3 is our language of choice, you should be fluent with that language to apply
* [https://git-scm.com Git] is our version control system of choice, you should be familiar with it to apply
* additional prerequisites depend on the project you will work on; check project descriptions for details

=== Before you apply ===

Here are the steps you should follow before applying, to make sure you have a good grasp of what we are doing at Software Heritage and how we do it:

# Follow our [https://docs.softwareheritage.org/devel/getting-started.html getting started guide]: it will make sure you can locally run a (small) copy of the archive and ingest source code into it
# Create an account our [https://forge.softwareheritage.org development forge]
# Familiarize yourself with our [[Code review in Phabricator|code review workflow]]
# Make a simple change to any one of our [https://docs.softwareheritage.org/devel/ software components] and submit it as a [https://forge.softwareheritage.org/differential/ diff] for code review, following the above workflow. [[Easy hacks]] and [https://forge.softwareheritage.org/project/view/20/ Web UI] issues are good options for what to fix, but feel free to submit any patch you think it might be useful.

=== What to include in your application ===

Make sure that your application includes the following information:

* Describe the '''specific project''' you want to work on. What do you want to achieve? Why is it important? Why is it useful for Software Heritage? The project might be one of the project ideas that we have prepared, or something else entirely that you want to contribute to Software Heritage. Your source code archival pet peeve, surprise us!
* Detail your '''work plan''': a brief description of how you plan to go about your project, including a list of ''deliverables'' and a ''timeline'' of when do you expect them to be available.
* Include a reference to '''the diff''' you submitted before applying (see the "Before you apply" section above).

== Ideas list ==

Below you can find a list of project ideas that are good options for a reasonably sized GSoC project.
They are just suggestion though, don't feel obliged to pick one of them if there is nothing that fits your taste and abilities.
Feel free to propose something else that you are excited about and that contributes to improve the Software Heritage archive: we will be happy to consider it!

=== Increase archive coverage ===

Software Heritage aims to archive '''all the software'''. We naturally started with the place where most of the software is easily available today: git repositories on GitHub.

As Software Heritage grows, we're incrementally trying to increase the coverage of the archive by expanding the sources from which we archive software. We built ways of archiving things like mercurial repositories, Debian packages, pypi bundles, etc.

Expanding the coverage of the archive has two different components:

1. Create '''origin listers'''. Listers are pieces of code that crawl the APIs of Software Forges[https://en.wikipedia.org/wiki/Forge_(software)] (like Bitbucket, Gitorious, Sourceforge, NPM...) and return a list of the software available in it. The documentation on listers is here: https://docs.softwareheritage.org/devel/swh-lister/index.html

2. Create '''loaders'''. Loaders take a bundle of software (tarball, git repository, Python package, ...) and load it into Software Heritage, by adapting it so that it matches our uniform data model[https://docs.softwareheritage.org/devel/swh-model/data-model.html].

In a few words, a lister can be a way of asking "what are all the repositories available on npm.org?", while a loader would be "how do I load the NPM package I downloaded into Software Heritage?".

Writing a lister or a loader is a great way to contribute to Software Heritage by expanding its coverage! We have a list of software sources we would like to archive here[https://wiki.softwareheritage.org/wiki/Suggestion_box:_source_code_to_add], but you're free to suggest more.

=== Mine information from archived content ===

'''TODO'''

=== Improve and extend the archive Web UI ===

In order to easily navigate into the archive content, a [https://forge.softwareheritage.org/source/swh-web/ web application] is currently developed.
So far it offers the following main features:
* programmatic access to the content of the archive via the [https://archive.softwareheritage.org/api/ Software Heritage API]
* in-browser navigation of the content of the archive via the [https://archive.softwareheritage.org/browse/ Software Heritage browse UI]

There is still numerous improvements and new features to add to that web application, for instance:
* improve overall design
* improve navigation for mobile browsers
* add new source code search criteria and improve the search interface
* implement new developer oriented features: source file history, blame interface, ...
* improve web application [https://www.w3.org/WAI/ accessibility]

If you are interested in web development and want to contribute to the application
enabling users to navigate in the biggest public source code archive collected so far,
feel free to apply.

== Contact ==

GSoC students are encouraged to get in touch with the Software Heritage community using the standard development communication channels, i.e.:

* the #swh-devel IRC channel on [https://freenode.net Freenode]
* the [https://sympa.inria.fr/sympa/info/swh-devel swh-devel mailing list]

See our [https://www.softwareheritage.org/community/developers/ development information page] for more details.

== Timeline ==

See the official [https://developers.google.com/open-source/gsoc/timeline Google Summer of Code timeline].