Gitea: Enable repository code search #31

New Issue

Arnd Marijnissen · 2023-02-14T11:32:15+01:00

Arnd Marijnissen commented

2023-02-14 11:32:15 +01:00

Currently we have no way of supporting code-search without risking running out of disk really quickly.
The documentation lists that code-search requires about 6 times the disk-space of the repo involved.

Code-search has a few controls we can use to govern its functioning; however, not enough to quite do what we'd like it to do (by default).

It permits 'bleve' (local) indexing or using ElasticSearch
It does not have controls in Gitea (yet) that'd permit restricting it to 'interesting'/'relevant' repositories.

In the interest of scalability, it'd seem logical to consider ElasticSearch backend as being the most useful. It might be possible to filter out the data we want indexed on the side of ElasticSearch. This'd permit us to enable 'REPO_INDEXER' on the short term, at the expense of it indexing things that we dont store the data for. Ultimately, a control in Gitea/repos to disable indexing would be the best solution.

Currently we have no way of supporting code-search without risking running out of disk really quickly. The documentation lists that code-search requires about 6 times the disk-space of the repo involved. Code-search has a few controls we can use to govern its functioning; however, not enough to quite do what we'd like it to do (by default). * It permits 'bleve' (local) indexing or using ElasticSearch * It does not have controls in Gitea (yet) that'd permit restricting it to 'interesting'/'relevant' repositories. In the interest of scalability, it'd seem logical to consider ElasticSearch backend as being the most useful. It might be possible to filter out the data we want indexed on the side of ElasticSearch. This'd permit us to enable 'REPO_INDEXER' on the short term, at the expense of it indexing things that we dont *store* the data for. Ultimately, a control in Gitea/repos to disable indexing would be the best solution.

❤️ 1

Arnd Marijnissen added the

labels 2023-02-14 11:32:15 +01:00

Arnd Marijnissen referenced this issue

2023-02-14 11:44:27 +01:00

Configuration of Production Server #3

techknowlogick commented

2023-02-14 19:35:54 +01:00

Tracked in the following upstream issue: https://github.com/go-gitea/gitea/issues/22842

Brecht Van Lommel commented

2024-05-24 21:14:21 +02:00

Gitea 1.22 add support for code search without an index. I'm not sure how fast it is on a big repository, may be worth testing at some point.

Sergey Sharybin commented

2024-05-27 09:58:35 +02:00

Is the code search for non-blender repositories already enabled?
Just to understand why the task explicitly mentions blender/ repositories.

Is the code search for non-blender repositories already enabled? Just to understand why the task explicitly mentions `blender/` repositories.

Brecht Van Lommel changed title from ~~Deployment: Permit CodeSearch for blender/ repos~~ to Deployment: Enable repository code search

2024-05-27 10:33:40 +02:00

Brecht Van Lommel commented

2024-05-27 10:37:15 +02:00

Code search is not enabled anywhere. It explicitly mentioned blender/, but the more general thing is that we don't want to be indexing forks, as Gitea treats every one separately. There is a Gitea option for that now.

Code search is not enabled anywhere. It explicitly mentioned `blender/`, but the more general thing is that we don't want to be indexing forks, as Gitea treats every one separately. There is a Gitea option for that now.

Sergey Sharybin commented

2024-05-27 10:44:53 +02:00

So just to be sure I understand it correctly, it is now possible to enable indexing of all public repos within blender/, infrastructure/, studio/, and extensions/ without enabling search for the forks of those repos?

So just to be sure I understand it correctly, it is now possible to enable indexing of all public repos within `blender/`, `infrastructure/`, `studio/`, and `extensions/` without enabling search for the forks of those repos?

Brecht Van Lommel commented

2024-05-27 10:51:31 +02:00

There's this Gitea option, where I think we'd only be enable sources and templates.

REPO_INDEXER_REPO_TYPES = sources,forks,mirrors,templates

But maybe we don't need to enable indexing of search is acceptable without it. It uses git grep which is pretty fast for me locally, no idea if it will be ok on the server though.

There's this Gitea option, where I think we'd only be enable `sources` and `templates`. ``` REPO_INDEXER_REPO_TYPES = sources,forks,mirrors,templates ``` But maybe we don't need to enable indexing of search is acceptable without it. It uses `git grep` which is pretty fast for me locally, no idea if it will be ok on the server though.

Bart van der Braak added this to the DevOps Progress Board project 2024-07-16 12:59:13 +02:00

Bart van der Braak changed title from ~~Deployment: Enable repository code search~~ to Gitea: Enable repository code search

2024-07-17 15:18:22 +02:00

Bart van der Braak closed this issue

2024-08-14 10:31:13 +02:00

Bart van der Braak reopened this issue

2024-08-14 10:35:20 +02:00

Sign in to join this conversation.