Improve project search engine (GSoC task) - Revision history

Vlorentz: /* Potential mentors */ Remove zack from mentors

2021-03-11T09:41:46Z

Potential mentors: Remove zack from mentors

← Older revision		Revision as of 09:41, 11 March 2021
Line 30:		Line 30:
	== Potential mentors ==		== Potential mentors ==

	* Stefano Zacchiroli <zack@upsilon.cc> (zack on [[IRC]])
	* Valentin Lorentz (vlorentz on [[IRC]])		* Valentin Lorentz (vlorentz on [[IRC]])
	* Vincent Sellier (vsellier on [[IRC]])		* Vincent Sellier (vsellier on [[IRC]])

Vlorentz: /* Potential mentors */ add IRC nicks

2021-03-02T10:33:13Z

Potential mentors: add IRC nicks

← Older revision		Revision as of 10:33, 2 March 2021
Line 30:		Line 30:
	== Potential mentors ==		== Potential mentors ==

	* Stefano Zacchiroli <zack@upsilon.cc>		* Stefano Zacchiroli <zack@upsilon.cc> (zack on [[IRC]])
	* Valentin Lorentz		* Valentin Lorentz (vlorentz on [[IRC]])
	* Vincent Sellier		* Vincent Sellier (vsellier on [[IRC]])

	== Other relevant (but independent) tasks ==		== Other relevant (but independent) tasks ==

Vlorentz at 10:06, 2 March 2021

2021-03-02T10:06:44Z

← Older revision		Revision as of 10:06, 2 March 2021
Line 26:		Line 26:

	* Python 3 and Git are a must to work on any Software Heritage project		* Python 3 and Git are a must to work on any Software Heritage project
	* either ElasticSearch (if you want to improve the existing search component)		* either ElasticSearch (if you want to improve the existing search component) or an alternative (if you want to rewrite it with a different backend)
	or an alternative (if you want to rewrite it with a different backend)

	== Potential mentors ==		== Potential mentors ==

Vlorentz: add skills & mentors

2021-03-02T10:06:33Z

add skills & mentors

← Older revision		Revision as of 10:06, 2 March 2021
Line 11:		Line 11:

	This service is a MVP (Minimum Viable Product) that was written to replace an		This service is a MVP (Minimum Viable Product) that was written to replace an
	older service, based on ~~Postgresql~~, which was too slow.		older service, based on PostgreSQL, which was too slow.
	So there is a lot of room for improvement in terms of adding features and		So there is a lot of room for improvement in terms of adding features and
	making results more relevant.		making results more relevant.
Line 22:		Line 22:
	Depending on your preferences, this can be either purely backend changes,		Depending on your preferences, this can be either purely backend changes,
	or may involve changes to the web interface to use these new features.		or may involve changes to the web interface to use these new features.

			== Desirable skills ==

			* Python 3 and Git are a must to work on any Software Heritage project
			* either ElasticSearch (if you want to improve the existing search component)
			or an alternative (if you want to rewrite it with a different backend)

			== Potential mentors ==

			* Stefano Zacchiroli <zack@upsilon.cc>
			* Valentin Lorentz
			* Vincent Sellier

	== Other relevant (but independent) tasks ==		== Other relevant (but independent) tasks ==

Vlorentz: /* Relevant (but independent) tasks */ change section name

2021-02-10T12:24:16Z

Relevant (but independent) tasks: change section name

← Older revision		Revision as of 12:24, 10 February 2021
Line 23:		Line 23:
	or may involve changes to the web interface to use these new features.		or may involve changes to the web interface to use these new features.

	== ~~Relevant~~ (but independent) tasks ==		== Other relevant (but independent) tasks ==

	This task is only about searching on the existing Elasticsearch database;		This task is only about searching on the existing Elasticsearch database;

Vlorentz at 12:24, 10 February 2021

2021-02-10T12:24:02Z

← Older revision		Revision as of 12:24, 10 February 2021
Line 1:		Line 1:
			== Introduction ==

	The [https://archive.softwareheritage.org/ homepage of the Software Heritage archive]		The [https://archive.softwareheritage.org/ homepage of the Software Heritage archive]
	features a small search engine, that searched in project URLs and project metadata.		features a small search engine, that searched in project URLs and project metadata.
Line 5:		Line 7:
	By the time GSoC starts, it will be implemented by a very small Python service		By the time GSoC starts, it will be implemented by a very small Python service
	(under 100 lines of code) backed by ElasticSearch.		(under 100 lines of code) backed by ElasticSearch.

			== Task description ==

	This service is a MVP (Minimum Viable Product) that was written to replace an		This service is a MVP (Minimum Viable Product) that was written to replace an
Line 18:		Line 22:
	Depending on your preferences, this can be either purely backend changes,		Depending on your preferences, this can be either purely backend changes,
	or may involve changes to the web interface to use these new features.		or may involve changes to the web interface to use these new features.

			== Relevant (but independent) tasks ==

	This task is only about searching on the existing Elasticsearch database;		This task is only about searching on the existing Elasticsearch database;
Line 23:		Line 29:
	and [[Mine information from external sources (GSoC task)]] to fill this		and [[Mine information from external sources (GSoC task)]] to fill this
	database; but those are completely independent tasks.		database; but those are completely independent tasks.

			This database only contains project URLs and metadata, not source code.
			Source code search is more complex, but is available as an
			[[Source code search engine prototype (internship)\|internship topic]]

	[[Category:GSoC task]]		[[Category:GSoC task]]

Vlorentz: add links to the indexing tasks

2021-02-10T12:20:55Z

add links to the indexing tasks

← Older revision		Revision as of 12:20, 10 February 2021
Line 18:		Line 18:
	Depending on your preferences, this can be either purely backend changes,		Depending on your preferences, this can be either purely backend changes,
	or may involve changes to the web interface to use these new features.		or may involve changes to the web interface to use these new features.

			This task is only about searching on the existing Elasticsearch database;
			you may also be interested in [[Mine information from archived content (GSoC task)]]
			and [[Mine information from external sources (GSoC task)]] to fill this
			database; but those are completely independent tasks.

	[[Category:GSoC task]]		[[Category:GSoC task]]

Vlorentz: add § about web ui

2021-02-10T12:16:51Z

add § about web ui

← Older revision		Revision as of 12:16, 10 February 2021
Line 15:		Line 15:
	instead of simply doing a full-text search on the entire metadata;		instead of simply doing a full-text search on the entire metadata;
	but we are open to suggestions.		but we are open to suggestions.

			Depending on your preferences, this can be either purely backend changes,
			or may involve changes to the web interface to use these new features.

	[[Category:GSoC task]]		[[Category:GSoC task]]

Vlorentz: Created page with "The [https://archive.softwareheritage.org/ homepage of the Software Heritage archive] features a small search engine, that searched in project URLs and project metadata. Proje..."

2021-02-10T12:09:57Z

Created page with "The [https://archive.softwareheritage.org/ homepage of the Software Heritage archive] features a small search engine, that searched in project URLs and project metadata. Proje..."

New page

The [https://archive.softwareheritage.org/ homepage of the Software Heritage archive]
features a small search engine, that searched in project URLs and project metadata.
Project metadata includes name, description, authors, etc.

By the time GSoC starts, it will be implemented by a very small Python service
(under 100 lines of code) backed by ElasticSearch.

This service is a MVP (Minimum Viable Product) that was written to replace an
older service, based on Postgresql, which was too slow.
So there is a lot of room for improvement in terms of adding features and
making results more relevant.

Most of the features we have in mind are to
[https://forge.softwareheritage.org/T1523 allow finer search on project metadata],
instead of simply doing a full-text search on the entire metadata;
but we are open to suggestions.

[[Category:GSoC task]]