This is a continuation of #9743. I believe this makes the entire chart
compatible with `runAsNonRoot`, as long as you disable the "permission
fix" container.
The actual changes for the two services are quite different:
* Kvrocks already runs as a non-root user, but it specifies that user as
a string, which is incompatible with `runAsNonRoot`. Override it with an
explicit `runAsUser` setting.
I have submitted a patch for this upstream, so this workaround will not
be necessary for future versions of Kvrocks.
* Vector runs as root. However, it seems that (in our configuration, at
least) it can be forced to run with a non-root UID without any
difficulties, so do that. A few other changes are in support of that:
* I changed the port from 80 to 8282, since unprivileged users are
usually unable to listen on ports < 1024. With Docker this isn't a
problem, but K8s often uses different container runtimes. I chose 8282,
since we already use this port in the development configuration.
* I set `data_dir` to `/vector-data-dir` in the config file. By default,
this setting is set to `/var/lib/vector`, which is only writeable by
root. `/vector-data-dir` is mounted as a tmpfs volume into the
container, therefore it has 777 permissions.
TBH, I'm not sure if Vector actually writes anything to this directory
in our configuration, but this gives me more peace of mind.
To maintain consistency between Compose and Helm deployments, I also
changed the UID/GID in the Compose file, and added a `/vector-data-dir`
volume.
- Added support for consensus task and consensus job merging (API and
UI)
- Added simple consensus settings
- Added server tests
- Added new `consensus` RQ queue and worker
- Updated skeleton comparisons: hidden points now also contribute to the
skeleton similarity. Only visibility is taken into account for invisible
points
Limitations:
- Merging is supported for all annotations except 2d and 3d cuboids. 3d
tasks are not supported
- Annotation groups are not supported (each annotation is considered
separate in a group)
- Polygons and masks are not interchangeable (each type is compared only
with the same type)
Co-authored-by: Kirill Lakhov <kirill.lakhov@cvat.ai>
<!-- Raise an issue to propose your change
(https://github.com/opencv/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution
guide](https://opencv.github.io/cvat/docs/contributing/). -->
<!-- Provide a general summary of your changes in the Title above -->
### Motivation and context
<!-- Why is this change required? What problem does it solve? If it
fixes an open
issue, please link to the issue here. Describe your changes in detail,
add
screenshots. -->
These types of data have different characteristics and we have different
expectations on them:
* job queues are small and we'd rather not lose them (although losing
them is not fatal);
* cached chunks are large and we don't care if we lose them.
We currently store both in KeyDB, which has shown itself to not be
especially reliable. A few times we've had to clear the KeyDB store due
to data corruption, which destroyed the queues as well. While we'll
probably end up replacing KeyDB with something else, it would still be
useful to have the ability to just clear the cache volume without taking
out the job queues in the process.
As a solution to this, add a Redis service to be used only for the
queues (and potentially for other small data items). Using the original
Redis instead of KeyDB should also help with reliability (at least as
far as the job queues are concerned).
### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->
I checked the CVAT can still start using the development environment
instructions, the Compose file and the Helm chart.
### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [x] I have created a changelog fragment <!-- see top comment in
CHANGELOG.md -->
- ~~[ ] I have updated the documentation accordingly~~
- ~~[ ] I have added tests to cover my changes~~
- ~~[ ] I have linked related issues (see [GitHub docs](
https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~
- ~~[ ] I have increased versions of npm packages if it is necessary
([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning),
[cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning),
[cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning)
and
[cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~
### License
- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the
project.
Feel free to contact the maintainers if that's a concern.
The advantages of this are as follows:
* It's much easier for a developer to use one `docker compose up`
command to bring everything up than to run a custom command for each
service.
* We eliminate possible divergence of configuration (e.g. versions,
command-line parameters) between what we actually use and what's listed
in the documentation.
* It makes it easier to update the developer guide if new dependencies
are introduced.
* And speaking of new dependencies, we have KeyDB now, which hasn't been
added to the dev guide.
The disadvantage is that we have to run an extra copy of the CVAT
server, because otherwise OPA can't fetch its rules. I don't think it's
a significant issue, since it doesn't prevent you from debugging
anything.
<!-- Raise an issue to propose your change
(https://github.com/opencv/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution
guide](https://opencv.github.io/cvat/docs/contributing/). -->
<!-- Provide a general summary of your changes in the Title above -->
### Motivation and context
<!-- Why is this change required? What problem does it solve? If it
fixes an open
issue, please link to the issue here. Describe your changes in detail,
add
screenshots. -->
Working on #7245, I realized that I don't want to add another custom
command for running Redis in the development environment to the dev
guide. So I wanted to remove the custom commands entirely.
### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->
By manually following the updated instructions.
### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- ~~[ ] I have created a changelog fragment~~ <!-- see top comment in
CHANGELOG.md -->
- [x] I have updated the documentation accordingly
- ~~[ ] I have added tests to cover my changes~~
- [x] I have linked related issues (see [GitHub docs](
https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- ~~[ ] I have increased versions of npm packages if it is necessary
([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning),
[cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning),
[cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning)
and
[cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~
### License
- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the
project.
Feel free to contact the maintainers if that's a concern.
The server part of #6039
- Added support for Ground Truth jobs in a task
- Added support for job creation and removal (only Ground Truth jobs can
be created or removed in a task)
- Added a component to autocompute quality metrics for a task
- Added tests
- Fixed https://github.com/opencv/cvat/issues/5971 (both parts - the
outside problem and the manifest problem, the manifest part fix is also
available in #6216)
Co-authored-by: klakhov <kirill.9992@gmail.com>
Co-authored-by: Roman Donchenko <roman@cvat.ai>
Co-authored-by: Kirill Sizov <kirill.sizov@cvat.ai>
The policy that mandated this is no longer relevant now that CVAT is no
longer developed by Intel. Moreover, the source code included was not
even complete (it didn't contain Python or NPM packages).
This saves ~1.6 GB in the unpacked image (and probably a bunch of build
time too, but I didn't measure it).
* Configured CI workflow for public images
* updated license headers
* align list item indent rule for remarklint and prettier
* updated changelog
Co-authored-by: Nikita Manovich <nikita.manovich@intel.com>
Roman Donchenko