Software Heritage Wiki - User contributions [en]

Category:Web site

2016-05-01T10:31:01Z

Rdicosmo: Created page with "Category:Infrastructure"

[[Category:Infrastructure]]

WG/Ethical and Legal Issues and Environment

2016-04-04T16:45:57Z

Rdicosmo: First stable version of the ELIE working group charter and expected deliverables.

= Working group on Ethical and Legal Issues and Environment =
== Charter ==
=== Mission ===
The Software Heritage archive collects the source code of software projects
from a variety of origins. Each of these origins may provide metadata
describing different characteristics of ''the project'' of which this source
code is part, and potentially containing information on ''the people'' that
take part in it.

The goal of the ELIE working group is to explore and analyze the ''ethical'' and ''legal''
issues that may arise related to the collection and aggregation of the
information contained in the Software Heritage archive.

The ELIE working group will propose ''policy documents'' for guiding the
operation of the Software Heritage archive and network of peers, and
contribute to develop a consensus on the best practices, that may vary
according to the geographical location of the peer nodes.

=== Duration ===
This working group is open ended.

=== Expected outcomes ===
''Explore and analyse'' the ''ethical'' and ''legal'' issues relevant for Software Heritage.

''Monitor'' the best practices used in other initiatives for handling these issues.

''Develop policy documents'' proposing the best practices for handling these issues, and in
particular for handling sensitive of personal data, and take down notices.

''Develop and Evolve'' a set of minimal common commitments for the peer nodes
joining the Software Heritage network, that will be collected in a ''Software Heritage Peer Charter''.

=== Related working groups ===
This working group is related to: [[Distribution_Replication_and_Query | Distribution, Replication and Query (DIREQ)]], [[Metadata_and_Linked_Data | Metadata and Linked Data (MELD)]], [[Scientific_API| Scientific API (SAPI)]]

== Team contact(s) ==
* [http://www.dicosmo.org Roberto Di Cosmo]

* [https://upsilon.cc/~zack Stefano Zacchiroli]

== Documents ==
Documents produced by the working group will be listed in this section.

== Connections ==
Active or planned connections to other initiatives, and activities will be listed in this section.

== Infrastructure ==
=== Mailing list ===
* https://sympa.inria.fr/sympa/info/elie-wg-swh

WG/Scientific API

2016-04-04T16:09:56Z

Rdicosmo: First stable version of the SAPI working group charter and expected deliverables.

= Working group on Scientific APIs =
== Charter ==
=== Mission ===
The data collected by Software Heritage is harvested from a diverse set of origins,
and organised into a data model that provides a uniform view of all the source code,
and its development history, independently of the data model of their origins.

This uniformity of access to all the contents turns the Software Heritage
archive into a research corpus which is unprecedented, and uniquely
positioned for enabling a variety of scientific applications.

The goal ot the SAPI working group is to interface with the research community,
elicit the expected functionalities, and help identifying those that are
general enough to be incorporated in the Software Heritage data access API,
separating them from specific functionalities that are best implemented as
part of applications built on top of Software Heritage.

=== Duration ===
How long the group is expected to stay in operation; includes date of creation

=== Expected outcomes ===
''Monitor'' the needs of the research communities that want to use the Software Heritage corpus.

''Contribute'' to the Software Heritage ''data access API'', proposing extensions corresponding
to the functionalities that are best implemented inside the Software Heritage services,
as opposed to those that are best implemented in the client services.

''Raise awareness'' of the relevance of the Software Heritage corpus for research,
and encourage research centers interested in accessing it to become full nodes in the
Software Heritage network and host a local copy of the data.

=== Related working groups ===
This working group is related to: [[Modeling_and_Ingesting_Version_control_systems | Modeling and Ingesting Version control systems (MIV)]], [[Distribution_Replication_and_Query | Distribution, Replication and Query (DIREQ)]].

== Team contact(s) ==
* [http://www.dicosmo.org Roberto Di Cosmo]

* [https://upsilon.cc/~zack Stefano Zacchiroli]

== Documents ==
Documents produced by the working group will be listed in this section.

== Connections ==
Active or planned connections to other initiatives, and activities will be listed in this section.

== Infrastructure ==
=== Mailing list ===
* https://sympa.inria.fr/sympa/info/sapi-wg-swh

WG/Distribution Replication and Query

2016-04-03T20:04:00Z

Rdicosmo: First stable version of the DIREQ working group charter and expected deliverables.

= Working group on Distribution, Replication and Query =
== Charter ==
=== Mission ===
<blockquote><em>…let us save what remains: not by vaults and locks which fence them from the public eye and use in consigning them to the waste of time, but by such a multiplication of copies, as shall place them beyond the reach of accident.</em><br/>Thomas Jefferson, February 18, 1791</blockquote>

One of Software Heritage's main missions is to ensure that the source code
assets will be preserved in the long term. There are a variety of threats to
digital information, ranging from operational failures to physical accidents,
from human error to malicious attacks, from technical obsolescence to legal
incertainty.

The most promising approach to withstand all these challenges is to ensure we
build the Software Heritage archive on an infrastructure that is distributed
and replicated in all respects:

* ''technically'': a variety of different technology

* ''geographically'': on different continents

* ''administratively'': under different control structures

* ''legally'': under different legal systems

The main goal of the DIREQ working group is to monitor and evaluate existing
and forecoming approaches to distributed resilient archival, and to develop and
evolve an API allowing the Software Heritage network of peers to abstract from
the particular technologies used to implement the storage backends and the
metadata database.
This API will need to address the issues related to reading and writing
information on the storage backend, and also updating and querying the
metadata database, which may be also distributed.

=== Duration ===
This working group is open ended.

=== Expected outcomes ===
The main expected outcomes are listed below.

''A common API for the distributed object storage'' abstracting away the details
of the different underlying technologies that may be adopted by the network of
peers. The solution proposed must allow several distinct technologies to be
in operation at the same time, monitoring the degree of replication achieved.

''State-of-the-art approaches to distributed metadata databases'' allowing to
implement queries and updates on the database holding all the Software Heritage
metadata. The append-only nature of this database may have implications on the
impact of the well known CAP issues for distributed databases.

''Monitoring and evaluation'' of existing and forecoming approaches to
distributed resilient archival and databases.

''Awareness''
The DIREQ working group will establish the relevant connections in order to
raise awareness among all the interested parties.

=== Milestones ===

== Team contact(s) ==
* [http://www.dicosmo.org Roberto Di Cosmo]

* [https://upsilon.cc/~zack Stefano Zacchiroli]

== Documents ==
Documents produced by the working group will be listed in this section.

== Connections ==
Active or planned connections to other initiatives, and activities will be listed in this section.

== Infrastructure ==
=== Mailing list ===
* https://sympa.inria.fr/sympa/info/direq-wg-swh

WG/Source Discovery and Ingestion

2016-04-03T19:16:10Z

Rdicosmo:

= Working group on Source Discovery and Ingestion =
== Charter ==
Collecting the source code that is publicly available is an essential part of Software Heritage's mission.
In order to fulfill this mission, we will need to discover, harvest, and keep up to date, content coming from a very diverse
set of possible origins like:

* several kinds of [https://en.wikipedia.org/wiki/Comparison_of_source_code_hosting_facilities source code repositories], including:
** mainstream development platforms, like GitHub, BitBucket, Sourceforge or Codeplex;
** institutional forges, like Inria's, Cenatic's or Adullact's FusionForge;
** community repositories like Debian's FusionForge or Joomla's gForge, Gnu's [http://savannah.gnu.org/ Savane], and the [http://www.apache.org/dyn/closer.cgi/ Apache] or [https://git.eclipse.org/c/ Eclipse] custom repositories;
** a variety of different source code archives, ranging from the [ftp://ftp.gnu.org Gnu ftp server] to individual web pages

This is a challenging task, and in order to succeed, the involvement of a large community is needed.

=== Mission ===
The SODI working group's mission is to foster ''the development'' and
''the adoption'' of software components that can, for a given origin, make it
''discoverable'', and ''list'' its contents, both ''in its entirety'', which is
useful for a newly discovered origin, and ''incrementally'', which is useful
to keep the Software Heritage archive up to date with the origin's evolving
content.

=== Duration ===
This working group is open ended.

=== Expected outcomes ===
The main expected outcomes of the SODI working group are listed below:

''APIs for discovering and tracking origins''
The SODI working group will define and evolve, in collaboration with the Software Heritage core team, standard APIs for software components that can be plugged in the Software Heritage infrastructure to track a (class of) origin(s). Whenever possible, proactive mechanisms for informing the Software Heritage infrastructure of content updates, like event feeds, will be preferred to approaches that require periodic polling of an origin.

''Adoption''
The SODI working group will strive, whenever possible, to have such components incorporated in the upstream code of the forges, and adopted widely.

''Awareness''
The SODI working group will establish the relevant connections in order to raise awareness among all the interested parties.

=== Milestones ===
A first set of requirements for the API, and a preliminary draft of the API is
expected to emerge from the work performed to ensure that Inria's own forge(s)
are properly tracked in the Software Heritage architecture.

=== Related working groups ===
This working group is related to: [[Modeling_and_Ingesting_Version_control_systems | Modeling and Ingesting Version control systems (MIV)]]

== Team contact(s) ==
* [http://upsilon.cc/~zack/ Stefano Zacchiroli]

== Documents ==
Documents produced by the working group will be listed in this section.

== Connections ==
Active or planned connections to other initiatives, and activities, will be listed in this section.

== Infrastructure ==
=== Mailing list ===
* https://sympa.inria.fr/sympa/info/sodi-wg-swh

[[Category: technical]]
[[Category: ingestion]]

WARC

2016-03-19T11:31:39Z

Rdicosmo:

The '''WARC (Web ARChive) file format''' offers a convention for concatenating multiple resource records (data objects), each consisting of a set of simple text headers and an arbitrary data block into one long file.

It has been released as an international standard by ISO in May 2009.

== References ==

* [http://archive-access.sourceforge.net/warc/ WARC specifications], including early drafts
* [http://www.digitalpreservation.gov/formats/fdd/fdd000236.shtml WARC entry on digitalpreservation.gov]
* [http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=44717 ISO 28500:2009]
* [http://bibnum.bnf.fr/WARC/ WARC information on the BNF website]
* [https://en.wikipedia.org/wiki/Web_ARChive WARC on Wikipedia]

[[Category:File format]]
[[Category:Related standards]]

Category:Related standards

2016-03-19T11:28:47Z

Rdicosmo: Created page with "Standards related to Software Heritage Category:Related work"

Standards related to [[Software Heritage]]

[[Category:Related work]]

EGI

2015-11-30T21:20:52Z

Rdicosmo: Created page with "EGI was born in 2001 from a series of grid EU projects. Now 18 people based in Amsterdam (many Italians), to coordinate the federation of the data + compute structures of res..."

EGI was born in 2001 from a series of grid EU projects.

Now 18 people based in Amsterdam (many Italians), to coordinate the federation of the data + compute structures of research centers, universities, etc. in Europe (approx. 300). Ensures brokerage between users and providers of compute and data services.

Very involved in EU infrastructure programs.

== References ==

* project homepage: http://www.egi.eu/

[[Category:Related work]]
[[Category:Related research]]
[[Category:Related initiatives]]

Onedata.org

2015-11-30T21:17:28Z

Rdicosmo: Created page with "== Basic facts == Company selling solutions for distributed storage. Interesting technology for high throughput data access on a distributed filesystem. Connected to and sup..."

== Basic facts ==

Company selling solutions for distributed storage.
Interesting technology for high throughput data access on a distributed filesystem.

Connected to and supported by [[EGI]]

== References ==

* project homepage: http://www.onedata.org

[[Category:Related work]]
[[Category:Related research]]
[[Category:Related software]]

Related technology

2015-11-19T20:21:58Z

Rdicosmo: /* Scancode */

Pointers to technology related to [[Software Heritage]] that we might want to reuse to realize the project.

== Bitergia ==

* https://github.com/MetricsGrimoire
* https://metricsgrimoire.github.io/

== Markos EU project ==

* [[Markos]]

== [[GitHub]] license analysis ==

as an example to how list / analyze GitHub repos batchly

* https://github.com/copiesofcopies/github-license-analysis
* reference: http://www.softwarefreedom.org/resources/2013/lcs-slides-aaronw/
* mail thread with Aaron: Message-ID: <20150421123630.GA12492@upsilon.cc> (ask [[Zack]] for details)

== Scancode ==

* https://github.com/nexB/scancode-toolkit/
* debian-qa thread: https://lists.debian.org/CAHedpx8b-ijpBbB1mhbatt0jBy4D983rkaMLYG2F6J7bLf5rzg@mail.gmail.com

[[Category:Related software]]
[[Category:Related work]]

== Serchcode ==
* http://www.boyter.org/category/searchcode/ and https://searchcode.com/ provide full text code search over more than 20Billions lines of code, last work in October 2015, done by Ben E. Boyter in Sidney as a side project.

Software is pervasive in Science

2015-09-13T18:50:21Z

Rdicosmo:

Articles dealing with pervasive software in Science

* http://pirsquared.org/tmp/Science-2013-Joppa-814-5.pdf -- recent survey on the availability of Source Code in software used for Science, with many links to articles pointing out how Softare is Pervasive

[[Category:Related work]]
[[Category:Scientific articles]]

Category:Scientific articles

2015-09-13T18:45:26Z

Rdicosmo:

Scientific articles related to Software Heritage

* Reproducibility
* [[Software is pervasive in Science|Software is pervasive in Science]]
* Software as subject of study
* Software practice

Related reproducibility initiatives

2015-08-03T16:53:47Z

Rdicosmo: /* meetings/working groups/conferences */

== the recomputation manifesto ==
http://www.software.ac.uk/blog/2013-07-09-recomputation-manifesto

== reproducibility in computer science ==
* [http://reproducibility.cs.arizona.edu/ analysis of publications in operating systems papers]
* [http://link.springer.com/article/10.1007%2Fs10664-011-9181-9 analysis of replicability in MSR papers] - Barahona

== papers on "executable papers" ==

by Konrad Hinsen:
* ''A data and code model for reproducible research and executable papers'' http://www.sciencedirect.com/science/article/pii/S1877050911001190
* ''Software Development for Reproducible Research'' http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6664970

== meetings/working groups/conferences ==

* [http://cascimodot.fdpoisson.fr/node/51 Rencontre de Réflexion autour de la Recherche Reproductible (R4)] Kornad Hinsen

== run my code ==
http://www.runmycode.org/

[[Category:Related work]]
[[Category:Reproducibility]]

Category:Legal

2015-05-11T14:53:46Z

Rdicosmo: Created page with "All legal issues related to the Software Heritage project."

All legal issues related to the Software Heritage project.

Category:Foundations

2015-05-10T17:32:19Z

Rdicosmo: Created page with "This page lists all the foundations we know of that are of interest for Software Heritage"

This page lists all the foundations we know of that are of interest for Software Heritage

Internet Archive

2015-05-10T17:31:50Z

Rdicosmo:

The Internet Archive is a 501(c)(3) non-profit that is located in San Francisco. Founded in 1996 to build an Internet library. Its purposes include offering permanent access for researchers, historians, scholars, people with disabilities, and the general public to historical collections that exist in digital format.

* About: https://archive.org/About
* Spokesperson : [http://en.wikipedia.org/wiki/Brewster_Kahle Brewster Kahle]

[[Category:Related work]]
[[Category:Foundations]]

Internet Memory Foundation

2015-05-10T17:31:27Z

Rdicosmo:

The Internet Memory Foundation is a non-profit institution. It actively supports the preservation of Internet as a new media for heritage and cultural purpose. Currently involved in various [[EuropeanProjects|european research projects]] and is developing several technologies to support the growth and use of the Internet memory, like the [[archivethe.net|Archive the net]] social network mining tool.

* [http://internetmemory.org/ About]
* [http://internetmemory.org/en/index.php/projects/ Projects]

[[Category:Related work]]
[[Category:Foundations]]

Category:Funding

2015-05-10T07:03:58Z

Rdicosmo: Created page with "Pages in this categoy point to initiatives that have obtained funding, and funding schemas en general."

Pages in this categoy point to initiatives that have obtained funding, and funding schemas en general.