Compare commits
6 Commits
3b26ff9fc1
...
2bead6af27
| Author | SHA1 | Date |
|---|---|---|
|
|
2bead6af27 | 2 weeks ago |
|
|
9aeedb39b8 | 2 weeks ago |
|
|
076bebbedf | 2 weeks ago |
|
|
836d8600c6 | 2 weeks ago |
|
|
be9c9c6df1 | 2 weeks ago |
|
|
2a90c0cd11 | 2 weeks ago |
28 changed files with 1115 additions and 517 deletions
@ -1,8 +1,14 @@ |
|||||||
github.com/Jguer/aur v1.3.0 h1:skdjp/P9kB75TBaJmn9PKK/kCeA9QsgjdUrORZ3gldU= |
|
||||||
github.com/Jguer/aur v1.3.0/go.mod h1:F8Awo+WKzTxlXtNOO4pDQjMkePLZ+oMSbu+1fKLTTLo= |
|
||||||
github.com/Jguer/dyalpm v0.1.2 h1:Gl0+GDWBQmo3DSsfzTPnKqCwYqcroq0j6kAtsIUkpUw= |
github.com/Jguer/dyalpm v0.1.2 h1:Gl0+GDWBQmo3DSsfzTPnKqCwYqcroq0j6kAtsIUkpUw= |
||||||
github.com/Jguer/dyalpm v0.1.2/go.mod h1:FpcWwU1eYHVWMKmr/yHFqHYKsS+qGKCtk/FIXirj2MY= |
github.com/Jguer/dyalpm v0.1.2/go.mod h1:FpcWwU1eYHVWMKmr/yHFqHYKsS+qGKCtk/FIXirj2MY= |
||||||
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= |
||||||
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= |
||||||
github.com/ebitengine/purego v0.10.0 h1:QIw4xfpWT6GWTzaW5XEKy3HXoqrJGx1ijYHzTF0/ISU= |
github.com/ebitengine/purego v0.10.0 h1:QIw4xfpWT6GWTzaW5XEKy3HXoqrJGx1ijYHzTF0/ISU= |
||||||
github.com/ebitengine/purego v0.10.0/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ= |
github.com/ebitengine/purego v0.10.0/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ= |
||||||
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= |
||||||
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= |
||||||
|
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= |
||||||
|
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= |
||||||
github.com/urfave/cli/v3 v3.8.0 h1:XqKPrm0q4P0q5JpoclYoCAv0/MIvH/jZ2umzuf8pNTI= |
github.com/urfave/cli/v3 v3.8.0 h1:XqKPrm0q4P0q5JpoclYoCAv0/MIvH/jZ2umzuf8pNTI= |
||||||
github.com/urfave/cli/v3 v3.8.0/go.mod h1:ysVLtOEmg2tOy6PknnYVhDoouyC/6N42TMeoMzskhso= |
github.com/urfave/cli/v3 v3.8.0/go.mod h1:ysVLtOEmg2tOy6PknnYVhDoouyC/6N42TMeoMzskhso= |
||||||
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= |
||||||
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= |
||||||
|
|||||||
@ -1,2 +1,2 @@ |
|||||||
schema: spec-driven |
schema: spec-driven |
||||||
created: 2026-04-13 |
created: 2026-04-14 |
||||||
@ -0,0 +1,137 @@ |
|||||||
|
## Context |
||||||
|
|
||||||
|
Currently, `pkg/pacman/pacman.go` uses subprocess calls to query pacman for package existence: |
||||||
|
- `pacman -Qip <pkg>` to check local DB (per package) |
||||||
|
- `pacman -Sip <pkg>` to check sync repos (per package) |
||||||
|
|
||||||
|
For n packages, this spawns 2n subprocesses (up to ~300 for typical package lists). Each subprocess has fork/exec overhead, making this the primary performance bottleneck. |
||||||
|
|
||||||
|
The AUR queries are already batched (single HTTP POST with all package names), which is the desired pattern. |
||||||
|
|
||||||
|
## Goals / Non-Goals |
||||||
|
|
||||||
|
**Goals:** |
||||||
|
- Eliminate subprocess overhead for local/sync DB package lookups |
||||||
|
- Maintain batched AUR HTTP calls (single request per batch) |
||||||
|
- Track installed status per package in PackageInfo |
||||||
|
- Provide dry-run output showing exact packages to install/remove |
||||||
|
- Handle orphan cleanup correctly |
||||||
|
|
||||||
|
**Non-Goals:** |
||||||
|
- Parallel AUR builds (still sequential) |
||||||
|
- Custom pacman transaction handling (use system pacman) |
||||||
|
- Repository configuration changes |
||||||
|
- Package download/compile optimization |
||||||
|
|
||||||
|
## Decisions |
||||||
|
|
||||||
|
### 1. Use Jguer/dyalpm for DB access |
||||||
|
|
||||||
|
**Decision**: Use `github.com/Jguer/dyalpm` library instead of spawning subprocesses. |
||||||
|
|
||||||
|
**Rationale**: |
||||||
|
- Direct libalpm access (same backend as pacman) |
||||||
|
- Already Go-native with proper type safety |
||||||
|
- Supports batch operations via `GetPkgCache()` and `PkgCache()` iterators |
||||||
|
|
||||||
|
**Alternatives considered**: |
||||||
|
- Parse `pacman -Qs` output - fragile, still subprocess-based |
||||||
|
- Write custom libalpm bindings - unnecessary effort |
||||||
|
|
||||||
|
### 2. Single-pass package resolution algorithm |
||||||
|
|
||||||
|
**Decision**: Process all packages through local DB → sync DBs → AUR in one pass. |
||||||
|
|
||||||
|
``` |
||||||
|
For each package in collected state: |
||||||
|
1. Check local DB (batch lookup) → if found, mark Installed=true |
||||||
|
2. If not local, check all sync DBs (batch lookup per repo) |
||||||
|
3. If not in sync, append to AUR batch |
||||||
|
|
||||||
|
Batch query AUR with all remaining packages |
||||||
|
Throw error if any package not found in local/sync/AUR |
||||||
|
|
||||||
|
Collect installed status from local DB |
||||||
|
(Perform sync operations - skip in dry-run) |
||||||
|
(Mark ALL currently installed packages as deps - skip in dry-run) |
||||||
|
(Then mark collected state packages as explicit - skip in dry-run) |
||||||
|
(Cleanup orphans - skip in dry-run) |
||||||
|
Output summary |
||||||
|
``` |
||||||
|
|
||||||
|
**Rationale**: |
||||||
|
- Single iteration over packages |
||||||
|
- Batch DB lookups minimize libalpm calls |
||||||
|
- Clear error handling for missing packages |
||||||
|
- Consistent with existing behavior |
||||||
|
|
||||||
|
### 3. Batch local/sync DB lookup implementation |
||||||
|
|
||||||
|
**Decision**: For local DB, iterate `localDB.PkgCache()` once and build a map. For sync DBs, iterate each repo's `PkgCache()`. |
||||||
|
|
||||||
|
**Implementation**: |
||||||
|
```go |
||||||
|
// Build local package map in one pass |
||||||
|
localPkgs := make(map[string]bool) |
||||||
|
localDB.PkgCache().ForEach(func(pkg alpm.Package) error { |
||||||
|
localPkgs[pkg.Name()] = true |
||||||
|
return nil |
||||||
|
}) |
||||||
|
|
||||||
|
// Similarly for each sync DB |
||||||
|
for _, syncDB := range syncDBs { |
||||||
|
syncDB.PkgCache().ForEach(...) |
||||||
|
} |
||||||
|
``` |
||||||
|
|
||||||
|
**Rationale**: |
||||||
|
- O(n) iteration where n = total packages in DB (not n queries) |
||||||
|
- Single map construction, O(1) lookups per state package |
||||||
|
- libalpm iterators are already lazy, no additional overhead |
||||||
|
|
||||||
|
### 4. Dry-run behavior |
||||||
|
|
||||||
|
**Decision**: Dry-run outputs exact packages that would be installed/removed without making any system changes. |
||||||
|
|
||||||
|
**Implementation**: |
||||||
|
- Skip `pacman -Syu` call |
||||||
|
- Skip `pacman -D --asdeps` (mark all installed as deps) |
||||||
|
- Skip `pacman -D --asexplicit` (mark state packages as explicit) |
||||||
|
- Skip `pacman -Rns` orphan cleanup |
||||||
|
- Still compute what WOULD happen for output |
||||||
|
|
||||||
|
**Note on marking strategy**: |
||||||
|
Instead of diffing between before/after installed packages, we simply: |
||||||
|
1. After sync completes, run `pacman -D --asdeps` on ALL currently installed packages (this marks everything as deps) |
||||||
|
2. Then run `pacman -D --asexplicit` on the collected state packages (this overrides them to explicit) |
||||||
|
|
||||||
|
This is simpler and achieves the same result. |
||||||
|
|
||||||
|
## Risks / Trade-offs |
||||||
|
|
||||||
|
1. **[Risk]** dyalpm initialization requires root privileges |
||||||
|
- **[Mitigation]** This is same as pacman itself; if user can't run pacman, declpac won't work |
||||||
|
|
||||||
|
2. **[Risk]** libalpm state becomes stale if another pacman instance runs concurrently |
||||||
|
- **[Mitigation]** Use proper locking, rely on pacman's own locking mechanism |
||||||
|
|
||||||
|
3. **[Risk]** AUR packages still built sequentially |
||||||
|
- **[Acceptable]** Parallel AUR builds out of scope for this change |
||||||
|
|
||||||
|
4. **[Risk]** Memory usage for large package lists |
||||||
|
- **[Mitigation]** Package map is ~100 bytes per package; 10k packages = ~1MB |
||||||
|
|
||||||
|
## Migration Plan |
||||||
|
|
||||||
|
1. Add `github.com/Jguer/dyalpm` to go.mod |
||||||
|
2. Refactor `ValidatePackage()` to use dyalpm instead of subprocesses |
||||||
|
3. Add `Installed bool` to `PackageInfo` struct |
||||||
|
4. Implement new resolution algorithm in `categorizePackages()` |
||||||
|
5. Update `Sync()` and `DryRun()` to use new algorithm |
||||||
|
6. Test with various package combinations |
||||||
|
7. Verify output matches previous behavior |
||||||
|
|
||||||
|
## Open Questions |
||||||
|
|
||||||
|
- **Q**: Should we also use dyalpm for `GetInstalledPackages()`? |
||||||
|
- **A**: Yes, can use localDB.PkgCache().Collect() or iterate - aligns with overall approach |
||||||
@ -0,0 +1,35 @@ |
|||||||
|
## Why |
||||||
|
|
||||||
|
The current pacman implementation spawns multiple subprocesses per package (pacman -Qip, pacman -Sip) to check if packages exist in local/sync DBs or AUR. With many packages, this creates significant overhead. Using the Jguer/dyalpm library provides direct libalpm access for batch queries, eliminating subprocess overhead while maintaining the batched AUR HTTP calls. |
||||||
|
|
||||||
|
## What Changes |
||||||
|
|
||||||
|
- **Add dyalpm dependency**: Integrate Jguer/dyalpm library for direct libalpm access |
||||||
|
- **Batch local DB check**: Use `localDB.PkgCache()` to check all packages at once instead of per-package `pacman -Qip` |
||||||
|
- **Batch sync DB check**: Use `syncDBs[i].PkgCache()` to check all sync repos at once instead of per-package `pacman -Sip` |
||||||
|
- **Enhance PackageInfo**: Add `Installed bool` field to track if package is already installed |
||||||
|
- **New algorithm**: Implement unified package resolution flow: |
||||||
|
1. Batch check local DB for all packages |
||||||
|
2. Batch check sync DBs for remaining packages |
||||||
|
3. Batch query AUR for non-found packages |
||||||
|
4. Track installed status throughout |
||||||
|
5. Perform sync operations with proper marking |
||||||
|
6. Output summary of changes |
||||||
|
|
||||||
|
## Capabilities |
||||||
|
|
||||||
|
### New Capabilities |
||||||
|
|
||||||
|
- `batch-package-resolution`: Unified algorithm that batch-resolves packages from local DB → sync DBs → AUR with proper installed tracking |
||||||
|
- `dry-run-simulation`: Shows exact packages that would be installed/removed without making changes |
||||||
|
|
||||||
|
### Modified Capabilities |
||||||
|
|
||||||
|
- None - this is a pure optimization with no behavior changes visible to users |
||||||
|
|
||||||
|
## Impact |
||||||
|
|
||||||
|
- **Code**: `pkg/pacman/pacman.go` - refactored to use dyalpm |
||||||
|
- **Dependencies**: Add Jguer/dyalpm to go.mod |
||||||
|
- **APIs**: `ValidatePackage()` signature changes (returns installed status) |
||||||
|
- **Performance**: O(n) subprocess calls → O(1) for local/sync DB checks |
||||||
@ -0,0 +1,72 @@ |
|||||||
|
## ADDED Requirements |
||||||
|
|
||||||
|
### Requirement: Batch package resolution from local, sync, and AUR databases |
||||||
|
The system SHALL resolve packages in a single pass through local DB → sync DBs → AUR using batch operations to minimize subprocess/API calls. |
||||||
|
|
||||||
|
#### Scenario: Package exists in local DB |
||||||
|
- **WHEN** a package from collected state exists in the local database |
||||||
|
- **THEN** the system SHALL mark it as found, set `Installed=true`, and exclude it from AUR queries |
||||||
|
|
||||||
|
#### Scenario: Package exists in sync DB |
||||||
|
- **WHEN** a package from collected state does NOT exist in local DB but exists in ANY enabled sync database |
||||||
|
- **THEN** the system SHALL mark it as found, set `Installed=false`, and exclude it from AUR queries |
||||||
|
|
||||||
|
#### Scenario: Package exists only in AUR |
||||||
|
- **WHEN** a package from collected state does NOT exist in local or sync databases but exists in AUR |
||||||
|
- **THEN** the system SHALL mark it as found with `InAUR=true`, set `Installed=false`, and use the cached AUR info |
||||||
|
|
||||||
|
#### Scenario: Package not found anywhere |
||||||
|
- **WHEN** a package from collected state is NOT in local DB, NOT in any sync DB, and NOT in AUR |
||||||
|
- **THEN** the system SHALL return an error listing the unfound package(s) |
||||||
|
|
||||||
|
#### Scenario: Batch AUR query |
||||||
|
- **WHEN** multiple packages need AUR lookup |
||||||
|
- **THEN** the system SHALL make a SINGLE HTTP request to AUR RPC with all package names (existing behavior preserved) |
||||||
|
|
||||||
|
### Requirement: Efficient local DB lookup using dyalpm |
||||||
|
The system SHALL use dyalpm's `PkgCache()` iterator to build a lookup map in O(n) time, where n is total packages in local DB, instead of O(n*m) subprocess calls. |
||||||
|
|
||||||
|
#### Scenario: Build local package map |
||||||
|
- **WHEN** initializing package resolution |
||||||
|
- **THEN** the system SHALL iterate localDB.PkgCache() once and store all package names in a map for O(1) lookups |
||||||
|
|
||||||
|
#### Scenario: Check package in local map |
||||||
|
- **WHEN** checking if a package exists in local DB |
||||||
|
- **THEN** the system SHALL perform an O(1) map lookup instead of spawning a subprocess |
||||||
|
|
||||||
|
### Requirement: Efficient sync DB lookup using dyalpm |
||||||
|
The system SHALL use each sync DB's `PkgCache()` iterator to check packages across all enabled repositories. |
||||||
|
|
||||||
|
#### Scenario: Check package in sync DBs |
||||||
|
- **WHEN** a package is not found in local DB |
||||||
|
- **THEN** the system SHALL check all enabled sync databases using their iterators |
||||||
|
|
||||||
|
#### Scenario: Package found in multiple sync repos |
||||||
|
- **WHEN** a package exists in more than one sync repository (e.g., core and community) |
||||||
|
- **THEN** the system SHALL use the first match found |
||||||
|
|
||||||
|
### Requirement: Track installed status in PackageInfo |
||||||
|
The system SHALL include an `Installed bool` field in `PackageInfo` to indicate whether the package is currently installed. |
||||||
|
|
||||||
|
#### Scenario: Package is installed |
||||||
|
- **WHEN** a package exists in the local database |
||||||
|
- **THEN** `PackageInfo.Installed` SHALL be `true` |
||||||
|
|
||||||
|
#### Scenario: Package is not installed |
||||||
|
- **WHEN** a package exists only in sync DB or AUR (not in local DB) |
||||||
|
- **THEN** `PackageInfo.Installed` SHALL be `false` |
||||||
|
|
||||||
|
### Requirement: Mark installed packages as deps, then state packages as explicit |
||||||
|
After package sync completes, the system SHALL mark all installed packages as dependencies, then override the collected state packages to be explicit. This avoids diffing before/after states. |
||||||
|
|
||||||
|
#### Scenario: Mark all installed as deps |
||||||
|
- **WHEN** package sync has completed (non-dry-run) |
||||||
|
- **THEN** the system SHALL run `pacman -D --asdeps` to mark ALL currently installed packages as dependencies |
||||||
|
|
||||||
|
#### Scenario: Override state packages to explicit |
||||||
|
- **WHEN** all installed packages have been marked as deps |
||||||
|
- **THEN** the system SHALL run `pacman -D --asexplicit` on the collected state packages, overriding their dependency status |
||||||
|
|
||||||
|
#### Scenario: Dry-run skips marking |
||||||
|
- **WHEN** operating in dry-run mode |
||||||
|
- **THEN** the system SHALL NOT execute any `pacman -D` marking operations |
||||||
@ -0,0 +1,28 @@ |
|||||||
|
## ADDED Requirements |
||||||
|
|
||||||
|
### Requirement: Dry-run shows packages to install without making changes |
||||||
|
In dry-run mode, the system SHALL compute what WOULD happen without executing any pacman operations. |
||||||
|
|
||||||
|
#### Scenario: Dry-run lists packages to install |
||||||
|
- **WHEN** dry-run is enabled and packages need to be installed |
||||||
|
- **THEN** the system SHALL populate `Result.ToInstall` with all packages that would be installed (both sync and AUR) |
||||||
|
|
||||||
|
#### Scenario: Dry-run lists packages to remove |
||||||
|
- **WHEN** dry-run is enabled and orphan packages exist |
||||||
|
- **THEN** the system SHALL populate `Result.ToRemove` with the list of orphan packages and `Result.Removed` with the count |
||||||
|
|
||||||
|
#### Scenario: Dry-run skips pacman sync |
||||||
|
- **WHEN** dry-run is enabled |
||||||
|
- **THEN** the system SHALL NOT execute `pacman -Syu` for package installation |
||||||
|
|
||||||
|
#### Scenario: Dry-run skips explicit/deps marking |
||||||
|
- **WHEN** dry-run is enabled |
||||||
|
- **THEN** the system SHALL NOT execute `pacman -D --asdeps` or `pacman -D --asexplicit` |
||||||
|
|
||||||
|
#### Scenario: Dry-run skips orphan cleanup |
||||||
|
- **WHEN** dry-run is enabled |
||||||
|
- **THEN** the system SHALL NOT execute `pacman -Rns` for orphan removal |
||||||
|
|
||||||
|
#### Scenario: Dry-run outputs count summary |
||||||
|
- **WHEN** dry-run is enabled |
||||||
|
- **THEN** the system SHALL still compute and output `Result.Installed` and `Result.Removed` counts as if the operations had run |
||||||
@ -0,0 +1,51 @@ |
|||||||
|
## 1. Setup |
||||||
|
|
||||||
|
- [x] 1.1 Add `github.com/Jguer/dyalpm` to go.mod |
||||||
|
- [x] 1.2 Run `go mod tidy` to fetch dependencies |
||||||
|
|
||||||
|
## 2. Core Refactoring |
||||||
|
|
||||||
|
- [x] 2.1 Update `PackageInfo` struct to add `Installed bool` field |
||||||
|
- [x] 2.2 Create `Pac` struct with `alpm.Handle` instead of just aurCache |
||||||
|
- [x] 2.3 Implement `NewPac()` that initializes alpm handle and local/sync DBs |
||||||
|
|
||||||
|
## 3. Package Resolution Algorithm |
||||||
|
|
||||||
|
- [x] 3.1 Implement `buildLocalPkgMap()` - iterate localDB.PkgCache() to create lookup map |
||||||
|
- [x] 3.2 Implement `checkSyncDBs()` - iterate each sync DB's PkgCache() to find packages |
||||||
|
- [x] 3.3 Implement `resolvePackages()` - unified algorithm: |
||||||
|
- Step 1: Check local DB for all packages (batch) |
||||||
|
- Step 2: Check sync DBs for remaining packages (batch per repo) |
||||||
|
- Step 3: Batch query AUR for remaining packages |
||||||
|
- Step 4: Return error if any package unfound |
||||||
|
- Step 5: Track installed status from local DB |
||||||
|
|
||||||
|
## 4. Sync and DryRun Integration |
||||||
|
|
||||||
|
- [x] 4.1 Refactor `Sync()` function to use new resolution algorithm |
||||||
|
- [x] 4.2 Refactor `DryRun()` function to use new resolution algorithm |
||||||
|
- [x] 4.3 Preserve AUR batched HTTP calls (existing `fetchAURInfo`) |
||||||
|
- [x] 4.4 Preserve orphan cleanup logic (`CleanupOrphans()`) |
||||||
|
|
||||||
|
## 5. Marking Operations |
||||||
|
|
||||||
|
- [x] 5.1 Keep `MarkExplicit()` for marking state packages |
||||||
|
- [x] 5.2 After sync, run `pacman -D --asdeps` on ALL installed packages (simplifies tracking) |
||||||
|
- [x] 5.3 After deps marking, run `pacman -D --asexplicit` on collected state packages (overrides deps) |
||||||
|
- [x] 5.4 Skip marking operations in dry-run mode |
||||||
|
|
||||||
|
## 6. Cleanup and Output |
||||||
|
|
||||||
|
- [x] 6.1 Remove subprocess-based `ValidatePackage()` implementation |
||||||
|
- [x] 6.2 Remove subprocess-based `GetInstalledPackages()` implementation |
||||||
|
- [x] 6.3 Update output summary to show installed/removed counts |
||||||
|
- [x] 6.4 In dry-run mode, populate `ToInstall` and `ToRemove` lists |
||||||
|
|
||||||
|
## 7. Testing |
||||||
|
|
||||||
|
- [ ] 7.1 Test with packages in local DB only |
||||||
|
- [ ] 7.2 Test with packages in sync DBs only |
||||||
|
- [ ] 7.3 Test with AUR packages |
||||||
|
- [ ] 7.4 Test with missing packages (should error) |
||||||
|
- [ ] 7.5 Test dry-run mode output |
||||||
|
- [ ] 7.6 Test orphan detection and cleanup |
||||||
@ -0,0 +1,2 @@ |
|||||||
|
schema: spec-driven |
||||||
|
created: 2026-04-14 |
||||||
@ -0,0 +1,25 @@ |
|||||||
|
## Context |
||||||
|
|
||||||
|
The dyalpm library's `SyncDBs()` returns an empty slice because it doesn't automatically load sync databases from pacman.conf. This causes `resolvePackages()` to skip the sync DB search entirely and fall through to checking the AUR, which fails for official repo packages. |
||||||
|
|
||||||
|
## Goals / Non-Goals |
||||||
|
|
||||||
|
**Goals:** |
||||||
|
- Register sync databases so `resolvePackages()` can find official repo packages |
||||||
|
|
||||||
|
**Non-Goals:** |
||||||
|
- Modify package installation logic |
||||||
|
- Add support for custom repositories |
||||||
|
|
||||||
|
## Decisions |
||||||
|
|
||||||
|
1. **Register each repo manually** - After `handle.SyncDBs()` returns empty, loop through known repos (core, extra, multilib) and call `handle.RegisterSyncDB()` for each. |
||||||
|
|
||||||
|
2. **Use hardcoded repo list** - Arch Linux standard repos are core, extra, multilib. This matches pacman.conf. |
||||||
|
|
||||||
|
3. **Silent failure for missing repos** - If a repo isn't configured in pacman.conf, `RegisterSyncDB` will return a valid but empty DB. Filter by checking if `PkgCache()` has any packages. |
||||||
|
|
||||||
|
## Risks / Trade-offs |
||||||
|
|
||||||
|
- Hardcoded repo names may need updating if Arch adds/removes standard repos → Low risk, rare change |
||||||
|
- Repo registration could fail silently → Mitigated by checking PkgCache count |
||||||
@ -0,0 +1,21 @@ |
|||||||
|
## Why |
||||||
|
|
||||||
|
The program fails to find packages that exist in official repositories (like `cmake` in `extra`). The dyalpm library's `SyncDBs()` returns an empty list, so the code never searches the sync databases and falls through to checking the AUR, which also doesn't have the package. |
||||||
|
|
||||||
|
## What Changes |
||||||
|
|
||||||
|
- Register sync databases manually after initializing the dyalpm handle |
||||||
|
- Loop through known repos (core, extra, multilib) and call `RegisterSyncDB` for each |
||||||
|
- Handle the case where a repo might not be configured in pacman.conf |
||||||
|
|
||||||
|
## Capabilities |
||||||
|
|
||||||
|
### New Capabilities |
||||||
|
- None - this is a bug fix to existing functionality |
||||||
|
|
||||||
|
### Modified Capabilities |
||||||
|
- None |
||||||
|
|
||||||
|
## Impact |
||||||
|
|
||||||
|
- `pkg/pacman/pacman.go`: Modify `New()` function to register sync DBs after getting them from the handle |
||||||
@ -0,0 +1,7 @@ |
|||||||
|
## ADDED Requirements |
||||||
|
|
||||||
|
No new requirements - this is a bug fix to existing package resolution functionality. |
||||||
|
|
||||||
|
## MODIFIED Requirements |
||||||
|
|
||||||
|
No modified requirements. |
||||||
@ -0,0 +1,10 @@ |
|||||||
|
## 1. Implement sync DB registration |
||||||
|
|
||||||
|
- [x] 1.1 Modify Pac struct to include a method for registering sync DBs |
||||||
|
- [x] 1.2 In New(), after getting empty syncDBs from handle, loop through ["core", "extra", "multilib"] and call RegisterSyncDB for each |
||||||
|
- [x] 1.3 Filter out repos that have no packages (not configured in pacman.conf) |
||||||
|
|
||||||
|
## 2. Test the fix |
||||||
|
|
||||||
|
- [x] 2.1 Run `./declpac --dry-run --state <(echo "cmake")` and verify it resolves cmake from extra repo |
||||||
|
- [x] 2.2 Test with other official repo packages (e.g., "git" from extra, "base" from core) |
||||||
@ -0,0 +1,2 @@ |
|||||||
|
schema: spec-driven |
||||||
|
created: 2026-04-14 |
||||||
@ -0,0 +1,141 @@ |
|||||||
|
# Design: Refactor pkg Into Modular Packages |
||||||
|
|
||||||
|
## New Structure |
||||||
|
|
||||||
|
``` |
||||||
|
pkg/ |
||||||
|
├── pacman/ # write operations only |
||||||
|
│ └── pacman.go |
||||||
|
├── fetch/ # package resolution (NEW) |
||||||
|
│ └── fetch.go |
||||||
|
├── validation/ # DB freshness (existing, expand) |
||||||
|
│ └── validation.go |
||||||
|
├── output/ # (unchanged) |
||||||
|
│ └── output.go |
||||||
|
├── merge/ # (unchanged) |
||||||
|
│ └── merge.go |
||||||
|
└── input/ # (unchanged) |
||||||
|
└── input.go |
||||||
|
``` |
||||||
|
|
||||||
|
## Package Responsibilities |
||||||
|
|
||||||
|
### pkg/fetch (NEW) |
||||||
|
|
||||||
|
``` |
||||||
|
type Fetcher struct { |
||||||
|
handle dyalpm.Handle |
||||||
|
localDB dyalpm.Database |
||||||
|
syncDBs []dyalpm.Database |
||||||
|
aurCache map[string]AURPackage |
||||||
|
} |
||||||
|
|
||||||
|
func New() *Fetcher |
||||||
|
func (f *Fetcher) Close() error |
||||||
|
func (f *Fetcher) Resolve(packages []string) (map[string]*PackageInfo, error) |
||||||
|
func (f *Fetcher) ListOrphans() ([]string, error) |
||||||
|
``` |
||||||
|
|
||||||
|
Extracted from `pacman.go`: |
||||||
|
- `buildLocalPkgMap()` |
||||||
|
- `checkSyncDBs()` |
||||||
|
- `resolvePackages()` |
||||||
|
- AUR cache (`ensureAURCache()`, `fetchAURInfo()`) |
||||||
|
|
||||||
|
### pkg/pacman (REFACTORED) |
||||||
|
|
||||||
|
``` |
||||||
|
func Sync(packages []string) (*output.Result, error) |
||||||
|
func DryRun(packages []string) (*output.Result, error) |
||||||
|
func MarkAsExplicit(packages []string) error |
||||||
|
func MarkAllAsDeps() error |
||||||
|
func CleanupOrphans() (int, error) |
||||||
|
``` |
||||||
|
|
||||||
|
Write actions only: |
||||||
|
- `Sync()` - calls `Fetcher` for resolution, then pacman commands |
||||||
|
- `DryRun()` - calls `Fetcher.Resolve()` + `ListOrphans()` |
||||||
|
- `MarkAsExplicit()`, `MarkAllAsDeps()` |
||||||
|
- `CleanupOrphans()` - calls `ListOrphans()` then removes |
||||||
|
|
||||||
|
### pkg/validation (REFACTORED) |
||||||
|
|
||||||
|
``` |
||||||
|
func CheckDBFreshness() error |
||||||
|
``` |
||||||
|
|
||||||
|
Keep as-is: checks lock file age, auto-synces if stale. |
||||||
|
|
||||||
|
Remove from `pacman.go`: |
||||||
|
- `IsDBFresh()` - replaced by `CheckDBFreshness()` |
||||||
|
- `SyncDB()` - called by validation when stale |
||||||
|
|
||||||
|
### Orphan Deduplication |
||||||
|
|
||||||
|
```go |
||||||
|
// In fetch/fetch.go |
||||||
|
func (f *Fetcher) ListOrphans() ([]string, error) { |
||||||
|
cmd := exec.Command("pacman", "-Qdtq") |
||||||
|
// ... |
||||||
|
} |
||||||
|
|
||||||
|
// In pacman/pacman.go |
||||||
|
func CleanupOrphans() (int, error) { |
||||||
|
orphans, err := fetcher.ListOrphans() // reuse |
||||||
|
if err != nil || len(orphans) == 0 { |
||||||
|
return 0, nil |
||||||
|
} |
||||||
|
// ... remove |
||||||
|
} |
||||||
|
|
||||||
|
func DryRun(...) (*output.Result, error) { |
||||||
|
orphans, err := fetcher.ListOrphans() // reuse |
||||||
|
// ... |
||||||
|
} |
||||||
|
``` |
||||||
|
|
||||||
|
## Data Structures Move to fetch |
||||||
|
|
||||||
|
```go |
||||||
|
// In fetch/fetch.go |
||||||
|
type PackageInfo struct { |
||||||
|
Name string |
||||||
|
InAUR bool |
||||||
|
Exists bool |
||||||
|
Installed bool |
||||||
|
AURInfo *AURPackage |
||||||
|
syncPkg dyalpm.Package |
||||||
|
} |
||||||
|
|
||||||
|
type AURResponse struct { |
||||||
|
Results []AURPackage `json:"results"` |
||||||
|
} |
||||||
|
|
||||||
|
type AURPackage struct { |
||||||
|
Name string `json:"Name"` |
||||||
|
PackageBase string `json:"PackageBase"` |
||||||
|
Version string `json:"Version"` |
||||||
|
URL string `json:"URL"` |
||||||
|
} |
||||||
|
``` |
||||||
|
|
||||||
|
## Import Changes |
||||||
|
|
||||||
|
`pkg/pacman/fetch.go` will need: |
||||||
|
- `github.com/Jguer/dyalpm` |
||||||
|
- `github.com/Riyyi/declpac/pkg/output` |
||||||
|
|
||||||
|
`pkg/pacman/pacman.go` will need: |
||||||
|
- `github.com/Riyyi/declpac/pkg/fetch` |
||||||
|
- `github.com/Riyyi/declpac/pkg/output` |
||||||
|
|
||||||
|
## Dependencies to Add |
||||||
|
|
||||||
|
- New import: `pkg/fetch` in `pacman` package |
||||||
|
|
||||||
|
## No Changes To |
||||||
|
|
||||||
|
- CLI entry points |
||||||
|
- `output.Result` struct |
||||||
|
- `input.ReadPackages()` |
||||||
|
- `merge.Merge()` |
||||||
@ -0,0 +1,30 @@ |
|||||||
|
# Proposal: Refactor pkg Into Modular Packages |
||||||
|
|
||||||
|
## Summary |
||||||
|
|
||||||
|
Split monolithic `pkg/pacman/pacman.go` into focused packages: `fetch` (package resolution), `validation` (DB checks), and keep `pacman` for write actions only. Also deduplicate orphan detection logic. |
||||||
|
|
||||||
|
## Motivation |
||||||
|
|
||||||
|
`pacman.go` is 645 lines doing too much: |
||||||
|
- Package resolution (local + sync DBs + AUR) |
||||||
|
- DB freshness checks |
||||||
|
- Pacman write operations (sync, mark, clean) |
||||||
|
- Orphan listing/cleanup |
||||||
|
|
||||||
|
This violates single responsibility. Hard to test, reason about, or reuse. Also has duplication: |
||||||
|
- `validation.CheckDBFreshness()` and `pacman.IsDBFresh()` both check DB freshness |
||||||
|
- `listOrphans()` and `CleanupOrphans()` duplicate orphan detection |
||||||
|
|
||||||
|
## Scope |
||||||
|
|
||||||
|
- Extract package resolution to new `pkg/fetch/` |
||||||
|
- Move DB freshness to `pkg/validation/` (keep `CheckDBFreshness()`) |
||||||
|
- Keep only write actions in `pkg/pacman/` |
||||||
|
- Deduplicate orphan logic: one function for listing, reuse in cleanup and dry-run |
||||||
|
|
||||||
|
## Out of Scope |
||||||
|
|
||||||
|
- No new features |
||||||
|
- No API changes to CLI |
||||||
|
- No changes to `pkg/output/`, `pkg/merge/`, `pkg/input/` |
||||||
@ -0,0 +1,15 @@ |
|||||||
|
# Scope |
||||||
|
|
||||||
|
## In Scope |
||||||
|
|
||||||
|
- Extract package resolution from pacman.go to pkg/fetch |
||||||
|
- Deduplicate orphan listing |
||||||
|
- Keep pacman write operations in pacman package |
||||||
|
- Maintain existing CLI API |
||||||
|
|
||||||
|
## Out of Scope |
||||||
|
|
||||||
|
- New features |
||||||
|
- New package management backends (e.g., libalpm alternatives) |
||||||
|
- Config file changes |
||||||
|
- State file format changes |
||||||
@ -0,0 +1,38 @@ |
|||||||
|
# Tasks: Refactor pkg Into Modular Packages |
||||||
|
|
||||||
|
## Phase 1: Create pkg/fetch |
||||||
|
|
||||||
|
- [x] 1.1 Create `pkg/fetch/fetch.go` |
||||||
|
- [x] 1.2 Move `AURResponse`, `AURPackage`, `PackageInfo` structs to fetch |
||||||
|
- [x] 1.3 Move `buildLocalPkgMap()` to fetch as `Fetcher.buildLocalPkgMap()` |
||||||
|
- [x] 1.4 Move `checkSyncDBs()` to fetch as `Fetcher.checkSyncDBs()` |
||||||
|
- [x] 1.5 Move `resolvePackages()` to fetch as `Fetcher.Resolve()` |
||||||
|
- [x] 1.6 Move AUR cache methods (`ensureAURCache`, `fetchAURInfo`) to fetch |
||||||
|
- [x] 1.7 Add `New()` and `Close()` to Fetcher |
||||||
|
- [x] 1.8 Add `ListOrphans()` to Fetcher |
||||||
|
|
||||||
|
## Phase 2: Refactor pkg/pacman |
||||||
|
|
||||||
|
- [x] 2.1 Remove from pacman.go (now in fetch): |
||||||
|
- `buildLocalPkgMap()` |
||||||
|
- `checkSyncDBs()` |
||||||
|
- `resolvePackages()` |
||||||
|
- `ensureAURCache()` |
||||||
|
- `fetchAURInfo()` |
||||||
|
- `AURResponse`, `AURPackage`, `PackageInfo` structs |
||||||
|
- [x] 2.2 Remove `IsDBFresh()` and `SyncDB()` (use validation instead) |
||||||
|
- [x] 2.3 Update imports in pacman.go to include fetch package |
||||||
|
- [x] 2.4 Update `Sync()` to use `fetch.Fetcher` for resolution |
||||||
|
- [x] 2.5 Update `DryRun()` to call `fetcher.ListOrphans()` instead of duplicate call |
||||||
|
- [x] 2.6 Update `CleanupOrphans()` to call `fetcher.ListOrphans()` instead of duplicate call |
||||||
|
|
||||||
|
## Phase 3: Clean Up Validation |
||||||
|
|
||||||
|
- [x] 3.1 Keep `validation.CheckDBFreshness()` as-is |
||||||
|
- [x] 3.2 Remove any remaining DB freshness duplication |
||||||
|
|
||||||
|
## Phase 4: Verify |
||||||
|
|
||||||
|
- [x] 4.1 Run tests (if any exist) |
||||||
|
- [x] 4.2 Build: `go build ./...` |
||||||
|
- [x] 4.3 Verify CLI still works: test dry-run, sync, orphan cleanup |
||||||
@ -1,67 +0,0 @@ |
|||||||
## Context |
|
||||||
|
|
||||||
Current implementation in `pkg/pacman/pacman.go` spawns shell processes for pacman queries: |
|
||||||
|
|
||||||
1. **`ValidatePackage`**: Calls `pacman -Qip` then `pacman -Sip` (2 processes per package) to check if package exists in sync databases |
|
||||||
2. **`MarkExplicit`**: Calls `pacman -D --explicit <pkg>` individually per package (N processes for N packages) |
|
||||||
3. **AUR**: Called individually per package not found in pacman databases |
|
||||||
|
|
||||||
Performance issues scale with package count. No caching of package queries across calls. |
|
||||||
|
|
||||||
## Goals / Non-Goals |
|
||||||
|
|
||||||
**Goals:** |
|
||||||
- Use Jguer/dyalpm library to query pacman databases without spawning processes |
|
||||||
- Add in-memory cache for ALL package query results (pacman + AUR), valid for entire job duration |
|
||||||
- Batch `pacman -D --explicit` calls to single process for multiple packages |
|
||||||
- Batch AUR HTTP queries to single request for all packages not found in pacman |
|
||||||
|
|
||||||
**Non-Goals:** |
|
||||||
- Refactor AUR handling (already uses HTTP API - will batch it) |
|
||||||
- Add persistent cache (only job-duration in-memory) |
|
||||||
- Change other pacman operations (sync, cleanup) |
|
||||||
|
|
||||||
## Decisions |
|
||||||
|
|
||||||
1. **dyalpm over go-alpm**: dyalpm uses purego (no cgo), cleaner cross-compilation |
|
||||||
- Alternative: go-alpm (cgo-based) - rejected for compilation complexity |
|
||||||
|
|
||||||
2. **Unified cache in Pac struct**: Single cache map replaces separate aurCache |
|
||||||
- Alternative: keep separate caches - rejected, unnecessary complexity |
|
||||||
- Cache key: package name, value: PackageInfo struct |
|
||||||
|
|
||||||
3. **Batch MarkExplicit**: Accept `[]string` packages, pass all to single `pacman -D --explicit` call |
|
||||||
- Note: pacman -D accepts multiple packages in single call |
|
||||||
|
|
||||||
4. **Batch query strategy**: |
|
||||||
- Query all packages against dyalpm local DB → returns found[] |
|
||||||
- Query not-found against dyalpm sync DBs → returns found[] |
|
||||||
- Query remaining not-found against AUR HTTP (single batched request) |
|
||||||
- Current: pacman -Qip → pacman -Sip → AUR (per-package) |
|
||||||
- New: Batch dyalpm local → Batch dyalpm sync → Batch AUR |
|
||||||
|
|
||||||
5. **Fallback for dyalpm unavailable**: If dyalpm init fails, fall back to: |
|
||||||
- pacman -Qip for all packages (single process, capture all) |
|
||||||
- pacman -Sip for remaining (single process, capture all) |
|
||||||
- AUR HTTP batch for remaining (already batched) |
|
||||||
|
|
||||||
## Risks / Trade-offs |
|
||||||
|
|
||||||
- **Risk**: dyalpm requires libalpm.so.15 on system |
|
||||||
- Mitigation: Check at runtime, fallback to process spawn if missing |
|
||||||
|
|
||||||
- **Risk**: Cache invalidation edge cases (e.g., package installed during job) |
|
||||||
- Mitigation: Acceptable for declpac use case; user runs sync after config changes |
|
||||||
|
|
||||||
- **Risk**: AUR API batch size limits |
|
||||||
- Mitigation: Chunk large batches if AUR has limits (TBD in implementation) |
|
||||||
|
|
||||||
## Migration Plan |
|
||||||
|
|
||||||
1. Add dyalpm dependency to go.mod |
|
||||||
2. Refactor Pac struct: replace aurCache with unified pkgCache map |
|
||||||
3. Change ValidatePackage to ValidatePackages (slice input, batch processing) |
|
||||||
4. Update MarkExplicit to accept slice |
|
||||||
5. Batch AUR HTTP calls in ensureAURCache |
|
||||||
6. Update call sites in Sync(), categorizePackages, DryRun() |
|
||||||
7. Test with existing test suite |
|
||||||
@ -1,28 +0,0 @@ |
|||||||
## Why |
|
||||||
|
|
||||||
Current `ValidatePackage` spawns multiple `pacman` processes per package (-Qip, -Sip calls), causing performance issues when checking many packages. Additionally, `MarkExplicit` is called individually for each package, spawning a separate process per package. Finally, AUR HTTP calls are made per-package. Using the Jguer/dyalpm Go library eliminates process spawning overhead and enables efficient batch operations throughout. |
|
||||||
|
|
||||||
## What Changes |
|
||||||
|
|
||||||
- Replace shell-out to `pacman -Qip`/`pacman -Sip` in `ValidatePackage` with Jguer/dyalpm library calls |
|
||||||
- Add unified in-memory package cache to `Pac` struct (replaces existing aurCache), persisting for job duration |
|
||||||
- Replace individual `pacman -D --explicit` calls with single batch call |
|
||||||
- Batch AUR HTTP queries into single request for all packages not found in pacman databases |
|
||||||
- Add Jguer/dyalpm dependency |
|
||||||
|
|
||||||
## Capabilities |
|
||||||
|
|
||||||
### New Capabilities |
|
||||||
|
|
||||||
- **dyalpm-package-query**: Use dyalpm library for querying pacman package databases in batch, falling back to AUR HTTP batch for remaining packages. Results cached for job duration. |
|
||||||
- **batch-explicit-mark**: Batch multiple packages into single `pacman -D --explicit` call instead of per-package process spawn. |
|
||||||
|
|
||||||
### Modified Capabilities |
|
||||||
|
|
||||||
None. |
|
||||||
|
|
||||||
## Impact |
|
||||||
|
|
||||||
- **pkg/pacman/pacman.go**: Refactor `ValidatePackage` to accept slice, replace aurCache with unified pkgCache, refactor `MarkExplicit` to accept slice |
|
||||||
- **go.mod**: Add Jguer/dyalpm import |
|
||||||
- **Performance**: Reduced process spawns from O(n*2 + n AUR) to O(1) for ValidatePackages, O(n) to O(1) for MarkExplicit |
|
||||||
@ -1,16 +0,0 @@ |
|||||||
## ADDED Requirements |
|
||||||
|
|
||||||
### Requirement: MarkExplicit accepts multiple packages |
|
||||||
The MarkExplicit method SHALL accept a slice of package names and mark all of them as explicitly installed using a single pacman -D call. |
|
||||||
|
|
||||||
#### Scenario: Single package marked explicit |
|
||||||
- **WHEN** MarkExplicit is called with one package name |
|
||||||
- **THEN** pacman -D --explicit is called once with that package |
|
||||||
|
|
||||||
#### Scenario: Multiple packages marked explicit |
|
||||||
- **WHEN** MarkExplicit is called with multiple package names (e.g., ["pkg1", "pkg2"]) |
|
||||||
- **THEN** pacman -D --explicit is called once with all packages as arguments |
|
||||||
|
|
||||||
#### Scenario: Empty package slice |
|
||||||
- **WHEN** MarkExplicit is called with an empty slice |
|
||||||
- **THEN** no pacman call is made, method returns nil immediately |
|
||||||
@ -1,47 +0,0 @@ |
|||||||
## ADDED Requirements |
|
||||||
|
|
||||||
### Requirement: ValidatePackages uses dyalpm library with batch AUR fallback |
|
||||||
The ValidatePackages method SHALL accept a slice of package names, query them in batch using dyalpm, then batch query AUR for any not found in pacman databases. |
|
||||||
|
|
||||||
#### Scenario: All packages found in local database |
|
||||||
- **WHEN** ValidatePackages is called with packages that all exist in local pacman database |
|
||||||
- **THEN** returns a slice of PackageInfo with Exists=true, InAUR=false for each |
|
||||||
|
|
||||||
#### Scenario: Some packages found in sync database |
|
||||||
- **WHEN** ValidatePackages is called with packages where some are only in sync databases |
|
||||||
- **THEN** returns PackageInfo with Exists=true, InAUR=false for found packages |
|
||||||
|
|
||||||
#### Scenario: Some packages found in AUR |
|
||||||
- **WHEN** ValidatePackages is called and some packages are not in pacman databases |
|
||||||
- **THEN** those packages are batched to single AUR HTTP request, returns PackageInfo with InAUR=true |
|
||||||
|
|
||||||
#### Scenario: Packages not found anywhere |
|
||||||
- **WHEN** ValidatePackages is called and some packages not found in pacman or AUR |
|
||||||
- **THEN** returns PackageInfo with Exists=false, InAUR=false for those packages |
|
||||||
|
|
||||||
### Requirement: Unified package cache |
|
||||||
The Pac struct SHALL maintain a single cache map that stores all package query results (both pacman and AUR), replacing the separate aurCache. |
|
||||||
|
|
||||||
#### Scenario: Package already cached |
|
||||||
- **WHEN** ValidatePackages is called for a package already queried in current job |
|
||||||
- **THEN** cached result returned without any dyalpm or AUR calls |
|
||||||
|
|
||||||
#### Scenario: New Pac instance |
|
||||||
- **WHEN** New() creates a new Pac instance |
|
||||||
- **THEN** cache is empty (fresh job) |
|
||||||
|
|
||||||
### Requirement: Single batch AUR call |
|
||||||
The AUR HTTP API SHALL be called once with all package names not found in pacman databases. |
|
||||||
|
|
||||||
#### Scenario: Multiple packages not in pacman |
|
||||||
- **WHEN** 3 packages not found in local or sync databases |
|
||||||
- **THEN** single AUR HTTP request with all 3 in arg[] params |
|
||||||
- **AND** results cached for all 3 |
|
||||||
|
|
||||||
### Requirement: Fallback to process spawn if dyalpm unavailable |
|
||||||
If dyalpm library is not available at runtime, ValidatePackages SHALL fall back to spawning pacman -Qip/-Sip processes in batch. |
|
||||||
|
|
||||||
#### Scenario: dyalpm unavailable |
|
||||||
- **WHEN** dyalpm initialization fails |
|
||||||
- **THEN** uses pacman -Qip for local, -Sip for sync, AUR HTTP batch as fallback |
|
||||||
- **AND** behavior is identical to using dyalpm |
|
||||||
@ -1,36 +0,0 @@ |
|||||||
## 1. Dependencies |
|
||||||
|
|
||||||
- [ ] 1.1 Add github.com/Jguer/dyalpm to go.mod |
|
||||||
- [ ] 1.2 Run go mod tidy to resolve dependencies |
|
||||||
|
|
||||||
## 2. Pac Struct Updates |
|
||||||
|
|
||||||
- [ ] 2.1 Add dyalpm handle field to Pac struct |
|
||||||
- [ ] 2.2 Add package info cache field (map[string]PackageInfo) to Pac struct |
|
||||||
- [ ] 2.3 Update New() to initialize dyalpm handle |
|
||||||
- [ ] 2.4 Update Close() to release dyalpm handle |
|
||||||
|
|
||||||
## 3. ValidatePackage Implementation |
|
||||||
|
|
||||||
- [ ] 3.1 Implement ValidatePackage using dyalpm LocalDB().Pkg() |
|
||||||
- [ ] 3.2 Query sync databases if not found locally |
|
||||||
- [ ] 3.3 Add caching logic to store/query results |
|
||||||
- [ ] 3.4 Keep AUR fallback for packages not in pacman repos |
|
||||||
|
|
||||||
## 4. MarkExplicit Updates |
|
||||||
|
|
||||||
- [ ] 4.1 Change MarkExplicit signature to accept []string (slice of packages) |
|
||||||
- [ ] 4.2 Implement single pacman -D --explicit call with all packages |
|
||||||
- [ ] 4.3 Handle empty slice case (return nil immediately) |
|
||||||
|
|
||||||
## 5. Call Site Updates |
|
||||||
|
|
||||||
- [ ] 5.1 Update Sync() to pass packages slice to MarkExplicit |
|
||||||
- [ ] 5.2 Ensure categorizePackages works with new ValidatePackage |
|
||||||
|
|
||||||
## 6. Testing |
|
||||||
|
|
||||||
- [ ] 6.1 Run existing tests to verify no regressions |
|
||||||
- [ ] 6.2 Test ValidatePackage with local, sync, and AUR packages |
|
||||||
- [ ] 6.3 Test MarkExplicit with single and multiple packages |
|
||||||
- [ ] 6.4 Verify cache behavior (second call returns cached result) |
|
||||||
@ -0,0 +1,345 @@ |
|||||||
|
package fetch |
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding/json" |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"net/http" |
||||||
|
"net/url" |
||||||
|
"os" |
||||||
|
"os/exec" |
||||||
|
"strings" |
||||||
|
"time" |
||||||
|
|
||||||
|
"github.com/Jguer/dyalpm" |
||||||
|
) |
||||||
|
|
||||||
|
var ( |
||||||
|
Root = "/" |
||||||
|
LockFile = "/var/lib/pacman/db.lock" |
||||||
|
AURInfoURL = "https://aur.archlinux.org/rpc?v=5&type=info" |
||||||
|
) |
||||||
|
|
||||||
|
type Fetcher struct { |
||||||
|
aurCache map[string]AURPackage |
||||||
|
handle dyalpm.Handle |
||||||
|
localDB dyalpm.Database |
||||||
|
syncDBs []dyalpm.Database |
||||||
|
} |
||||||
|
|
||||||
|
type PackageInfo struct { |
||||||
|
Name string |
||||||
|
InAUR bool |
||||||
|
Exists bool |
||||||
|
Installed bool |
||||||
|
AURInfo *AURPackage |
||||||
|
syncPkg dyalpm.Package |
||||||
|
} |
||||||
|
|
||||||
|
type AURResponse struct { |
||||||
|
Results []AURPackage `json:"results"` |
||||||
|
} |
||||||
|
|
||||||
|
type AURPackage struct { |
||||||
|
Name string `json:"Name"` |
||||||
|
PackageBase string `json:"PackageBase"` |
||||||
|
Version string `json:"Version"` |
||||||
|
URL string `json:"URL"` |
||||||
|
} |
||||||
|
|
||||||
|
func New() (*Fetcher, error) { |
||||||
|
start := time.Now() |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] Fetcher New: starting...\n") |
||||||
|
|
||||||
|
handle, err := dyalpm.Initialize(Root, "/var/lib/pacman") |
||||||
|
if err != nil { |
||||||
|
return nil, fmt.Errorf("failed to initialize alpm: %w", err) |
||||||
|
} |
||||||
|
|
||||||
|
localDB, err := handle.LocalDB() |
||||||
|
if err != nil { |
||||||
|
handle.Release() |
||||||
|
return nil, fmt.Errorf("failed to get local database: %w", err) |
||||||
|
} |
||||||
|
|
||||||
|
syncDBs, err := handle.SyncDBs() |
||||||
|
if err != nil { |
||||||
|
handle.Release() |
||||||
|
return nil, fmt.Errorf("failed to get sync databases: %w", err) |
||||||
|
} |
||||||
|
|
||||||
|
if len(syncDBs) == 0 { |
||||||
|
syncDBs, err = registerSyncDBs(handle) |
||||||
|
if err != nil { |
||||||
|
handle.Release() |
||||||
|
return nil, fmt.Errorf("failed to register sync databases: %w", err) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
fmt.Fprintf(os.Stderr, "[debug] Fetcher New: done (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
return &Fetcher{ |
||||||
|
aurCache: make(map[string]AURPackage), |
||||||
|
handle: handle, |
||||||
|
localDB: localDB, |
||||||
|
syncDBs: syncDBs, |
||||||
|
}, nil |
||||||
|
} |
||||||
|
|
||||||
|
func (f *Fetcher) Close() error { |
||||||
|
if f.handle != nil { |
||||||
|
f.handle.Release() |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (f *Fetcher) GetAURPackage(name string) (AURPackage, bool) { |
||||||
|
pkg, ok := f.aurCache[name] |
||||||
|
return pkg, ok |
||||||
|
} |
||||||
|
|
||||||
|
func (f *Fetcher) BuildLocalPkgMap() (map[string]interface{}, error) { |
||||||
|
localPkgs, err := f.buildLocalPkgMap() |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
result := make(map[string]interface{}) |
||||||
|
for k, v := range localPkgs { |
||||||
|
result[k] = v |
||||||
|
} |
||||||
|
return result, nil |
||||||
|
} |
||||||
|
|
||||||
|
func registerSyncDBs(handle dyalpm.Handle) ([]dyalpm.Database, error) { |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] registerSyncDBs: starting...\n") |
||||||
|
|
||||||
|
repos := []string{"core", "extra", "multilib"} |
||||||
|
var dbs []dyalpm.Database |
||||||
|
|
||||||
|
for _, repo := range repos { |
||||||
|
db, err := handle.RegisterSyncDB(repo, 0) |
||||||
|
if err != nil { |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
count := 0 |
||||||
|
db.PkgCache().ForEach(func(pkg dyalpm.Package) error { |
||||||
|
count++ |
||||||
|
return nil |
||||||
|
}) |
||||||
|
|
||||||
|
if count > 0 { |
||||||
|
dbs = append(dbs, db) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
fmt.Fprintf(os.Stderr, "[debug] registerSyncDBs: done (%d dbs)\n", len(dbs)) |
||||||
|
return dbs, nil |
||||||
|
} |
||||||
|
|
||||||
|
func (f *Fetcher) buildLocalPkgMap() (map[string]dyalpm.Package, error) { |
||||||
|
start := time.Now() |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] buildLocalPkgMap: starting...\n") |
||||||
|
|
||||||
|
localPkgs := make(map[string]dyalpm.Package) |
||||||
|
|
||||||
|
err := f.localDB.PkgCache().ForEach(func(pkg dyalpm.Package) error { |
||||||
|
localPkgs[pkg.Name()] = pkg |
||||||
|
return nil |
||||||
|
}) |
||||||
|
|
||||||
|
if err != nil { |
||||||
|
return nil, fmt.Errorf("failed to iterate local package cache: %w", err) |
||||||
|
} |
||||||
|
|
||||||
|
fmt.Fprintf(os.Stderr, "[debug] buildLocalPkgMap: done (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
return localPkgs, nil |
||||||
|
} |
||||||
|
|
||||||
|
func (f *Fetcher) checkSyncDBs(pkgNames []string) (map[string]dyalpm.Package, error) { |
||||||
|
start := time.Now() |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] checkSyncDBs: starting...\n") |
||||||
|
|
||||||
|
syncPkgs := make(map[string]dyalpm.Package) |
||||||
|
pkgSet := make(map[string]bool) |
||||||
|
for _, name := range pkgNames { |
||||||
|
pkgSet[name] = true |
||||||
|
} |
||||||
|
|
||||||
|
for _, db := range f.syncDBs { |
||||||
|
err := db.PkgCache().ForEach(func(pkg dyalpm.Package) error { |
||||||
|
if pkgSet[pkg.Name()] { |
||||||
|
if _, exists := syncPkgs[pkg.Name()]; !exists { |
||||||
|
syncPkgs[pkg.Name()] = pkg |
||||||
|
} |
||||||
|
} |
||||||
|
return nil |
||||||
|
}) |
||||||
|
if err != nil { |
||||||
|
return nil, fmt.Errorf("failed to iterate sync database %s: %w", db.Name(), err) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
fmt.Fprintf(os.Stderr, "[debug] checkSyncDBs: done (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
return syncPkgs, nil |
||||||
|
} |
||||||
|
|
||||||
|
func (f *Fetcher) Resolve(packages []string) (map[string]*PackageInfo, error) { |
||||||
|
start := time.Now() |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] Resolve: starting...\n") |
||||||
|
|
||||||
|
result := make(map[string]*PackageInfo) |
||||||
|
|
||||||
|
localPkgs, err := f.buildLocalPkgMap() |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] Resolve: local pkgs built (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
|
||||||
|
var notInLocal []string |
||||||
|
for _, pkg := range packages { |
||||||
|
if localPkg, ok := localPkgs[pkg]; ok { |
||||||
|
result[pkg] = &PackageInfo{ |
||||||
|
Name: pkg, |
||||||
|
Exists: true, |
||||||
|
InAUR: false, |
||||||
|
Installed: true, |
||||||
|
syncPkg: localPkg, |
||||||
|
} |
||||||
|
} else { |
||||||
|
notInLocal = append(notInLocal, pkg) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if len(notInLocal) > 0 { |
||||||
|
syncPkgs, err := f.checkSyncDBs(notInLocal) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] Resolve: sync db checked (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
|
||||||
|
var notInSync []string |
||||||
|
for _, pkg := range notInLocal { |
||||||
|
if syncPkg, ok := syncPkgs[pkg]; ok { |
||||||
|
result[pkg] = &PackageInfo{ |
||||||
|
Name: pkg, |
||||||
|
Exists: true, |
||||||
|
InAUR: false, |
||||||
|
Installed: false, |
||||||
|
syncPkg: syncPkg, |
||||||
|
} |
||||||
|
} else { |
||||||
|
notInSync = append(notInSync, pkg) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if len(notInSync) > 0 { |
||||||
|
f.ensureAURCache(notInSync) |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] Resolve: AUR cache ensured (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
|
||||||
|
var unfound []string |
||||||
|
for _, pkg := range notInSync { |
||||||
|
if aurInfo, ok := f.aurCache[pkg]; ok { |
||||||
|
result[pkg] = &PackageInfo{ |
||||||
|
Name: pkg, |
||||||
|
Exists: true, |
||||||
|
InAUR: true, |
||||||
|
Installed: false, |
||||||
|
AURInfo: &aurInfo, |
||||||
|
} |
||||||
|
} else { |
||||||
|
unfound = append(unfound, pkg) |
||||||
|
} |
||||||
|
} |
||||||
|
if len(unfound) > 0 { |
||||||
|
return nil, fmt.Errorf("package(s) not found: %s", strings.Join(unfound, ", ")) |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
fmt.Fprintf(os.Stderr, "[debug] Resolve: done (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
return result, nil |
||||||
|
} |
||||||
|
|
||||||
|
func (f *Fetcher) ensureAURCache(packages []string) { |
||||||
|
start := time.Now() |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] ensureAURCache: starting...\n") |
||||||
|
|
||||||
|
if len(packages) == 0 { |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
var uncached []string |
||||||
|
for _, pkg := range packages { |
||||||
|
if _, ok := f.aurCache[pkg]; !ok { |
||||||
|
uncached = append(uncached, pkg) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if len(uncached) == 0 { |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] ensureAURCache: done (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
f.fetchAURInfo(uncached) |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] ensureAURCache: done (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
} |
||||||
|
|
||||||
|
func (f *Fetcher) fetchAURInfo(packages []string) map[string]AURPackage { |
||||||
|
start := time.Now() |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] fetchAURInfo: starting...\n") |
||||||
|
|
||||||
|
result := make(map[string]AURPackage) |
||||||
|
|
||||||
|
if len(packages) == 0 { |
||||||
|
return result |
||||||
|
} |
||||||
|
|
||||||
|
v := url.Values{} |
||||||
|
for _, pkg := range packages { |
||||||
|
v.Add("arg[]", pkg) |
||||||
|
} |
||||||
|
|
||||||
|
resp, err := http.Get(AURInfoURL + "&" + v.Encode()) |
||||||
|
if err != nil { |
||||||
|
return result |
||||||
|
} |
||||||
|
defer resp.Body.Close() |
||||||
|
|
||||||
|
body, err := io.ReadAll(resp.Body) |
||||||
|
if err != nil { |
||||||
|
return result |
||||||
|
} |
||||||
|
|
||||||
|
var aurResp AURResponse |
||||||
|
if err := json.Unmarshal(body, &aurResp); err != nil { |
||||||
|
return result |
||||||
|
} |
||||||
|
|
||||||
|
for _, r := range aurResp.Results { |
||||||
|
f.aurCache[r.Name] = r |
||||||
|
result[r.Name] = r |
||||||
|
} |
||||||
|
|
||||||
|
fmt.Fprintf(os.Stderr, "[debug] fetchAURInfo: done (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
return result |
||||||
|
} |
||||||
|
|
||||||
|
func (f *Fetcher) ListOrphans() ([]string, error) { |
||||||
|
start := time.Now() |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] ListOrphans: starting...\n") |
||||||
|
|
||||||
|
cmd := exec.Command("pacman", "-Qdtq") |
||||||
|
orphans, err := cmd.Output() |
||||||
|
if err != nil { |
||||||
|
return nil, nil |
||||||
|
} |
||||||
|
|
||||||
|
list := strings.TrimSpace(string(orphans)) |
||||||
|
if list == "" { |
||||||
|
fmt.Fprintf(os.Stderr, "[debug] ListOrphans: done (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
return nil, nil |
||||||
|
} |
||||||
|
|
||||||
|
fmt.Fprintf(os.Stderr, "[debug] ListOrphans: done (%.2fs)\n", time.Since(start).Seconds()) |
||||||
|
return strings.Split(list, "\n"), nil |
||||||
|
} |
||||||
Loading…
Reference in new issue