From 9047c7d320bce3edec117808da5ca3efd60cdc39 Mon Sep 17 00:00:00 2001 From: Han-Wen Nienhuys Date: Wed, 29 Nov 2017 09:30:30 +0100 Subject: [PATCH] Documentation: describe NoteDb group storage Change-Id: I4e542399e8c55aafa4cca501989533d614ef6aca --- Documentation/config-groups.txt | 109 ++++++++++++++++++++++++++++++++ Documentation/index.txt | 3 +- Documentation/note-db.txt | 7 +- 3 files changed, 115 insertions(+), 4 deletions(-) create mode 100644 Documentation/config-groups.txt diff --git a/Documentation/config-groups.txt b/Documentation/config-groups.txt new file mode 100644 index 0000000000..4db4cb367c --- /dev/null +++ b/Documentation/config-groups.txt @@ -0,0 +1,109 @@ + += Gerrit Code Review - Groups + +== Overview + +In Gerrit, we assign permissions to groups of accounts. These groups +can be provided by an external system such as LDAP, but Gerrit also +has a group system built-in ("internal groups") + +Starting from 2.16, these internal groups are fully stored in +link:note-db.html[NoteDb]. + +A group is characterized by the following information: + +* list of members (accounts) +* list of subgroups +* properties + - visibleToAll + - group owner + +Groups are keyed by the following unique identifiers: + +* GroupID, the former database key (a sequential number) + +* UUID, an opaque identifier. Internal groups use a 40 byte hex string +as UUID + +* Name: Gerrit enforces that group names are unique + +== Storage format + +Group data is stored in the +link:config-accounts.html#all-users[`All-Users` repository]. For each +group, there is a ref, stored as a sharded UUID, e.g. + +---- + refs/groups/ef/deafbeefdeafbeefdeafbeefdeafbeefdeafbeef +---- + +The ref points to commits holding files. The files are + +* `members`, holding numeric account IDs of members, one per line +* `subgroups`, holding group UUIDs of subgroups, one per line +* `group.config`, holding further configuration. + +The `group.config` file follows the following format + +---- +[group] + name = + id = 42 + visibleToAll = false + description = + groupOwnerUuid = +---- + +Gerrit updates the ref for a group based on REST API calls, and the +commit log effectively forms an audit log which shows how group +membership evolved over time. + +To ensure uniqueness of the name, a separate ref +`refs/meta/group-names` contains a notemap, ie. a map represented as a +branch with a flat list of files. + +The format of this map is as follows: + +* keys are the normal SHA1 of the group name +* values are blobs that look like ++ +---- +[group] + name = + uuid = +---- + +To ensure uniqueness of the sequential ID, the ID for each new group +is taken from the sequence counter under `refs/sequences/groups`, +which works analogously to the ones for accounts and changes. + +== Visibility + +Group ownership together with `visibleToAll` determines visibility of +the groups in the REST API. + +Fetching a group ref is permitted to the group's owners that also have +READ permissions on the ref. For users that are not owners, the +permissions on the ref are ignored. In addition, anyone with the +link:access-control.html#capability_accessDatabase[Access Database] +capability can read all group refs. The `refs/meta/group-names` ref is +visible only to users with the +link:access-control.html#capability_accessDatabase[Access Database] +capability. + +== Pushing to group refs + +Validation on push for changes to the group ref is not implemented, so +pushes are rejected. Pushes that bypass Gerrit should be avoided since +the names, IDs and UUIDs must be internally consistent between all the +branches involved. In addition, group references should not be created +or deleted manually either. If you attempt any of these actions +anyway, don't forget to link:rest-api-groups.html#index-group[Index +Group] reindex the affected groups manually. + +== Replication + +In a replicated setting (eg. backups and or master/slave +configurations), all refs in the `All-Users` project must be copied +onto all replicas, including `refs/groups/*`, `refs/meta/group-names` +and `refs/sequences/groups`. diff --git a/Documentation/index.txt b/Documentation/index.txt index 24c538f919..51ce9d61bc 100644 --- a/Documentation/index.txt +++ b/Documentation/index.txt @@ -67,7 +67,8 @@ . link:config-auto-site-initialization.html[Automatic Site Initialization on Startup] . link:pgm-index.html[Server Side Administrative Tools] . link:note-db.html[NoteDb] -. link:config-accounts.html[Accounts] +. link:config-accounts.html[Accounts on NoteDb] +. link:config-groups.html[Groups on NoteDb] == Developer . Getting Started diff --git a/Documentation/note-db.txt b/Documentation/note-db.txt index 1a986e2e70..9035cdab5b 100644 --- a/Documentation/note-db.txt +++ b/Documentation/note-db.txt @@ -31,7 +31,10 @@ same repository as code changes. - Storing link:config-accounts.html[account data] is fully implemented in the 2.15 release. Account data is migrated automatically during the upgrade process by running `gerrit.war init`. -- Account and change metadata on the servers behind `googlesource.com` is fully +- Storing link:config-groups.html[group metadata] is fully implemented + for the 2.16 release. Group data is migrated automatically during + the upgrade process by running `gerrit.war init` +- Account, group and change metadata on the servers behind `googlesource.com` is fully migrated to NoteDb. In other words, if you use link:https://gerrit-review.googlesource.com/[gerrit-review], you're already using NoteDb. @@ -44,8 +47,6 @@ For an example NoteDb change, poke around at this one: == Future Work ("Gerrit 3.0") -- Storing group data is a work in progress. Like account data, it will be - migrated automatically. - NoteDb will be the only database format supported by Gerrit 3.0. The offline change data migration tool will be included in Gerrit 3.0, but online migration will only be available in the 2.x line.