Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

project config 'restricted.cluster.groups' not checked when deleting a cluster group #15118

Open
wideawakening opened this issue Mar 6, 2025 · 1 comment · May be fixed by #15119
Open

project config 'restricted.cluster.groups' not checked when deleting a cluster group #15118

wideawakening opened this issue Mar 6, 2025 · 1 comment · May be fixed by #15119
Assignees
@markylaing
Labels
Jira Triggers the synchronization of a GitHub issue in Jira

Comments

@wideawakening
Copy link
Member

wideawakening commented Mar 6, 2025 •
edited
Loading

problem

spotted this while trying to restore a cluster member.
if you could make the error bit more verbose it would help. logs point into a more clear direction on the culprit

ubuntu@hpc-02:~$ lxc cluster restore hpc-01
Are you sure you want to restore cluster member "hpc-01"? (yes/no) [default=no]: yes
Error: Migration API failure: Project isn't allowed to use this cluster member: "hpc-01"

ubuntu-desktop-remote is a stopped instance, on project stable which has a restricted.cluster.groups that does not exist anymore (it was deleted some days back)

ubuntu@hpc-02:~$ lxc monitor --pretty  | grep hpc-01
...
time="2025-03-06T08:56:23+01:00" level=debug msg="Handling API request" fingerprint=9198a46812cfa3434c18bf56f7e519bc083db03008a1527c5f9c003198e0ed73 ip="REDACTED.24.228:44572" method=POST protocol=cluster url="/https://github.com/1.0/instances/ubuntu-desktop-remote?project=stable&target=hpc-01"
time="2025-03-06T08:56:23+01:00" level=debug msg="Database error" err="Project isn't allowed to use this cluster member: \"hpc-01\""
time="2025-03-06T08:56:23+01:00" level=info msg="ID: 6b6f9060-306d-44a5-98ee-9c3003279461, Class: task, Description: Restoring cluster member" CreatedAt="2025-03-06 08:56:23.037108935 +0100 CET" Err="Migration API failure: Project isn't allowed to use this cluster member: \"hpc-01\"" Location=hpc-01 MayCancel=false Metadata="map[evacuation_progress:Migrating \"ubuntu-desktop-remote\" in project \"stable\" from \"hpc-02\"]" Resources="map[]" Status=Failure StatusCode=Failure UpdatedAt="2025-03-06 08:56:23.371603162 +0100 CET"


ubuntu@hpc-01:~$ lxc project show stable
name: stable
description: use-cases for validation purpose
config:
...
  restricted.cluster.groups: hardened-nodes
...
used_by:
...

ubuntu@hpc-01:~$ lxc cluster group list
+---------+-----------------------+---------+
|  NAME   |      DESCRIPTION      | MEMBERS |
+---------+-----------------------+---------+
| default | Default cluster group | 5       |
+---------+-----------------------+---------+

howto reproduce

  • create a cluster group with a node
  • create a project and assign group as restricted.cluster.groups
  • delete group and now project is corrupted, since it's not validating that is not in use

more specific reproduction of our use case ,though i did not test it fully:

  • create a cluster group with a node
  • create a project and assign group as restricted.cluster.groups
  • create an instance in that project.
  • stop it (did not check if problem happens also with running instances, or even without an instance)
  • evacuate node
  • delete group and now project is corrupted and will probably get the first error
  • try restore node
@markylaing
Copy link
Contributor

This is happening because the cluster group is only weakly referenced be the projects restricted.cluster.groups configuration. I suspect we will need to add a UsedBy field to the API type for cluster groups and disallow deletion if it is used by a project.

@markylaing markylaing self-assigned this Mar 6, 2025
@markylaing markylaing added the Jira Triggers the synchronization of a GitHub issue in Jira label Mar 6, 2025
@markylaing markylaing linked a pull request Mar 6, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@markylaing markylaing

Labels
Jira Triggers the synchronization of a GitHub issue in Jira
Projects
None yet
Milestone
No milestone
2 participants
@wideawakening @markylaing