Compare commits
6 Commits
3b26ff9fc1
...
2bead6af27
| Author | SHA1 | Date |
|---|---|---|
AI Bot
|
2bead6af27 | 2 weeks ago |
|
AI Bot
|
9aeedb39b8 | 2 weeks ago |
|
AI Bot
|
076bebbedf | 2 weeks ago |
|
AI Bot
|
836d8600c6 | 2 weeks ago |
|
AI Bot
|
be9c9c6df1 | 2 weeks ago |
|
AI Bot
|
2a90c0cd11 | 2 weeks ago |
23 changed files with 1116 additions and 322 deletions
@ -1,8 +1,14 @@
|
||||
github.com/Jguer/aur v1.3.0 h1:skdjp/P9kB75TBaJmn9PKK/kCeA9QsgjdUrORZ3gldU= |
||||
github.com/Jguer/aur v1.3.0/go.mod h1:F8Awo+WKzTxlXtNOO4pDQjMkePLZ+oMSbu+1fKLTTLo= |
||||
github.com/Jguer/dyalpm v0.1.2 h1:Gl0+GDWBQmo3DSsfzTPnKqCwYqcroq0j6kAtsIUkpUw= |
||||
github.com/Jguer/dyalpm v0.1.2/go.mod h1:FpcWwU1eYHVWMKmr/yHFqHYKsS+qGKCtk/FIXirj2MY= |
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= |
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= |
||||
github.com/ebitengine/purego v0.10.0 h1:QIw4xfpWT6GWTzaW5XEKy3HXoqrJGx1ijYHzTF0/ISU= |
||||
github.com/ebitengine/purego v0.10.0/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ= |
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= |
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= |
||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= |
||||
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= |
||||
github.com/urfave/cli/v3 v3.8.0 h1:XqKPrm0q4P0q5JpoclYoCAv0/MIvH/jZ2umzuf8pNTI= |
||||
github.com/urfave/cli/v3 v3.8.0/go.mod h1:ysVLtOEmg2tOy6PknnYVhDoouyC/6N42TMeoMzskhso= |
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= |
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= |
||||
|
||||
@ -0,0 +1,2 @@
|
||||
schema: spec-driven |
||||
created: 2026-04-14 |
||||
@ -0,0 +1,137 @@
|
||||
## Context |
||||
|
||||
Currently, `pkg/pacman/pacman.go` uses subprocess calls to query pacman for package existence: |
||||
- `pacman -Qip <pkg>` to check local DB (per package) |
||||
- `pacman -Sip <pkg>` to check sync repos (per package) |
||||
|
||||
For n packages, this spawns 2n subprocesses (up to ~300 for typical package lists). Each subprocess has fork/exec overhead, making this the primary performance bottleneck. |
||||
|
||||
The AUR queries are already batched (single HTTP POST with all package names), which is the desired pattern. |
||||
|
||||
## Goals / Non-Goals |
||||
|
||||
**Goals:** |
||||
- Eliminate subprocess overhead for local/sync DB package lookups |
||||
- Maintain batched AUR HTTP calls (single request per batch) |
||||
- Track installed status per package in PackageInfo |
||||
- Provide dry-run output showing exact packages to install/remove |
||||
- Handle orphan cleanup correctly |
||||
|
||||
**Non-Goals:** |
||||
- Parallel AUR builds (still sequential) |
||||
- Custom pacman transaction handling (use system pacman) |
||||
- Repository configuration changes |
||||
- Package download/compile optimization |
||||
|
||||
## Decisions |
||||
|
||||
### 1. Use Jguer/dyalpm for DB access |
||||
|
||||
**Decision**: Use `github.com/Jguer/dyalpm` library instead of spawning subprocesses. |
||||
|
||||
**Rationale**: |
||||
- Direct libalpm access (same backend as pacman) |
||||
- Already Go-native with proper type safety |
||||
- Supports batch operations via `GetPkgCache()` and `PkgCache()` iterators |
||||
|
||||
**Alternatives considered**: |
||||
- Parse `pacman -Qs` output - fragile, still subprocess-based |
||||
- Write custom libalpm bindings - unnecessary effort |
||||
|
||||
### 2. Single-pass package resolution algorithm |
||||
|
||||
**Decision**: Process all packages through local DB → sync DBs → AUR in one pass. |
||||
|
||||
``` |
||||
For each package in collected state: |
||||
1. Check local DB (batch lookup) → if found, mark Installed=true |
||||
2. If not local, check all sync DBs (batch lookup per repo) |
||||
3. If not in sync, append to AUR batch |
||||
|
||||
Batch query AUR with all remaining packages |
||||
Throw error if any package not found in local/sync/AUR |
||||
|
||||
Collect installed status from local DB |
||||
(Perform sync operations - skip in dry-run) |
||||
(Mark ALL currently installed packages as deps - skip in dry-run) |
||||
(Then mark collected state packages as explicit - skip in dry-run) |
||||
(Cleanup orphans - skip in dry-run) |
||||
Output summary |
||||
``` |
||||
|
||||
**Rationale**: |
||||
- Single iteration over packages |
||||
- Batch DB lookups minimize libalpm calls |
||||
- Clear error handling for missing packages |
||||
- Consistent with existing behavior |
||||
|
||||
### 3. Batch local/sync DB lookup implementation |
||||
|
||||
**Decision**: For local DB, iterate `localDB.PkgCache()` once and build a map. For sync DBs, iterate each repo's `PkgCache()`. |
||||
|
||||
**Implementation**: |
||||
```go |
||||
// Build local package map in one pass |
||||
localPkgs := make(map[string]bool) |
||||
localDB.PkgCache().ForEach(func(pkg alpm.Package) error { |
||||
localPkgs[pkg.Name()] = true |
||||
return nil |
||||
}) |
||||
|
||||
// Similarly for each sync DB |
||||
for _, syncDB := range syncDBs { |
||||
syncDB.PkgCache().ForEach(...) |
||||
} |
||||
``` |
||||
|
||||
**Rationale**: |
||||
- O(n) iteration where n = total packages in DB (not n queries) |
||||
- Single map construction, O(1) lookups per state package |
||||
- libalpm iterators are already lazy, no additional overhead |
||||
|
||||
### 4. Dry-run behavior |
||||
|
||||
**Decision**: Dry-run outputs exact packages that would be installed/removed without making any system changes. |
||||
|
||||
**Implementation**: |
||||
- Skip `pacman -Syu` call |
||||
- Skip `pacman -D --asdeps` (mark all installed as deps) |
||||
- Skip `pacman -D --asexplicit` (mark state packages as explicit) |
||||
- Skip `pacman -Rns` orphan cleanup |
||||
- Still compute what WOULD happen for output |
||||
|
||||
**Note on marking strategy**: |
||||
Instead of diffing between before/after installed packages, we simply: |
||||
1. After sync completes, run `pacman -D --asdeps` on ALL currently installed packages (this marks everything as deps) |
||||
2. Then run `pacman -D --asexplicit` on the collected state packages (this overrides them to explicit) |
||||
|
||||
This is simpler and achieves the same result. |
||||
|
||||
## Risks / Trade-offs |
||||
|
||||
1. **[Risk]** dyalpm initialization requires root privileges |
||||
- **[Mitigation]** This is same as pacman itself; if user can't run pacman, declpac won't work |
||||
|
||||
2. **[Risk]** libalpm state becomes stale if another pacman instance runs concurrently |
||||
- **[Mitigation]** Use proper locking, rely on pacman's own locking mechanism |
||||
|
||||
3. **[Risk]** AUR packages still built sequentially |
||||
- **[Acceptable]** Parallel AUR builds out of scope for this change |
||||
|
||||
4. **[Risk]** Memory usage for large package lists |
||||
- **[Mitigation]** Package map is ~100 bytes per package; 10k packages = ~1MB |
||||
|
||||
## Migration Plan |
||||
|
||||
1. Add `github.com/Jguer/dyalpm` to go.mod |
||||
2. Refactor `ValidatePackage()` to use dyalpm instead of subprocesses |
||||
3. Add `Installed bool` to `PackageInfo` struct |
||||
4. Implement new resolution algorithm in `categorizePackages()` |
||||
5. Update `Sync()` and `DryRun()` to use new algorithm |
||||
6. Test with various package combinations |
||||
7. Verify output matches previous behavior |
||||
|
||||
## Open Questions |
||||
|
||||
- **Q**: Should we also use dyalpm for `GetInstalledPackages()`? |
||||
- **A**: Yes, can use localDB.PkgCache().Collect() or iterate - aligns with overall approach |
||||
@ -0,0 +1,35 @@
|
||||
## Why |
||||
|
||||
The current pacman implementation spawns multiple subprocesses per package (pacman -Qip, pacman -Sip) to check if packages exist in local/sync DBs or AUR. With many packages, this creates significant overhead. Using the Jguer/dyalpm library provides direct libalpm access for batch queries, eliminating subprocess overhead while maintaining the batched AUR HTTP calls. |
||||
|
||||
## What Changes |
||||
|
||||
- **Add dyalpm dependency**: Integrate Jguer/dyalpm library for direct libalpm access |
||||
- **Batch local DB check**: Use `localDB.PkgCache()` to check all packages at once instead of per-package `pacman -Qip` |
||||
- **Batch sync DB check**: Use `syncDBs[i].PkgCache()` to check all sync repos at once instead of per-package `pacman -Sip` |
||||
- **Enhance PackageInfo**: Add `Installed bool` field to track if package is already installed |
||||
- **New algorithm**: Implement unified package resolution flow: |
||||
1. Batch check local DB for all packages |
||||
2. Batch check sync DBs for remaining packages |
||||
3. Batch query AUR for non-found packages |
||||
4. Track installed status throughout |
||||
5. Perform sync operations with proper marking |
||||
6. Output summary of changes |
||||
|
||||
## Capabilities |
||||
|
||||
### New Capabilities |
||||
|
||||
- `batch-package-resolution`: Unified algorithm that batch-resolves packages from local DB → sync DBs → AUR with proper installed tracking |
||||
- `dry-run-simulation`: Shows exact packages that would be installed/removed without making changes |
||||
|
||||
### Modified Capabilities |
||||
|
||||
- None - this is a pure optimization with no behavior changes visible to users |
||||
|
||||
## Impact |
||||
|
||||
- **Code**: `pkg/pacman/pacman.go` - refactored to use dyalpm |
||||
- **Dependencies**: Add Jguer/dyalpm to go.mod |
||||
- **APIs**: `ValidatePackage()` signature changes (returns installed status) |
||||
- **Performance**: O(n) subprocess calls → O(1) for local/sync DB checks |
||||
@ -0,0 +1,72 @@
|
||||
## ADDED Requirements |
||||
|
||||
### Requirement: Batch package resolution from local, sync, and AUR databases |
||||
The system SHALL resolve packages in a single pass through local DB → sync DBs → AUR using batch operations to minimize subprocess/API calls. |
||||
|
||||
#### Scenario: Package exists in local DB |
||||
- **WHEN** a package from collected state exists in the local database |
||||
- **THEN** the system SHALL mark it as found, set `Installed=true`, and exclude it from AUR queries |
||||
|
||||
#### Scenario: Package exists in sync DB |
||||
- **WHEN** a package from collected state does NOT exist in local DB but exists in ANY enabled sync database |
||||
- **THEN** the system SHALL mark it as found, set `Installed=false`, and exclude it from AUR queries |
||||
|
||||
#### Scenario: Package exists only in AUR |
||||
- **WHEN** a package from collected state does NOT exist in local or sync databases but exists in AUR |
||||
- **THEN** the system SHALL mark it as found with `InAUR=true`, set `Installed=false`, and use the cached AUR info |
||||
|
||||
#### Scenario: Package not found anywhere |
||||
- **WHEN** a package from collected state is NOT in local DB, NOT in any sync DB, and NOT in AUR |
||||
- **THEN** the system SHALL return an error listing the unfound package(s) |
||||
|
||||
#### Scenario: Batch AUR query |
||||
- **WHEN** multiple packages need AUR lookup |
||||
- **THEN** the system SHALL make a SINGLE HTTP request to AUR RPC with all package names (existing behavior preserved) |
||||
|
||||
### Requirement: Efficient local DB lookup using dyalpm |
||||
The system SHALL use dyalpm's `PkgCache()` iterator to build a lookup map in O(n) time, where n is total packages in local DB, instead of O(n*m) subprocess calls. |
||||
|
||||
#### Scenario: Build local package map |
||||
- **WHEN** initializing package resolution |
||||
- **THEN** the system SHALL iterate localDB.PkgCache() once and store all package names in a map for O(1) lookups |
||||
|
||||
#### Scenario: Check package in local map |
||||
- **WHEN** checking if a package exists in local DB |
||||
- **THEN** the system SHALL perform an O(1) map lookup instead of spawning a subprocess |
||||
|
||||
### Requirement: Efficient sync DB lookup using dyalpm |
||||
The system SHALL use each sync DB's `PkgCache()` iterator to check packages across all enabled repositories. |
||||
|
||||
#### Scenario: Check package in sync DBs |
||||
- **WHEN** a package is not found in local DB |
||||
- **THEN** the system SHALL check all enabled sync databases using their iterators |
||||
|
||||
#### Scenario: Package found in multiple sync repos |
||||
- **WHEN** a package exists in more than one sync repository (e.g., core and community) |
||||
- **THEN** the system SHALL use the first match found |
||||
|
||||
### Requirement: Track installed status in PackageInfo |
||||
The system SHALL include an `Installed bool` field in `PackageInfo` to indicate whether the package is currently installed. |
||||
|
||||
#### Scenario: Package is installed |
||||
- **WHEN** a package exists in the local database |
||||
- **THEN** `PackageInfo.Installed` SHALL be `true` |
||||
|
||||
#### Scenario: Package is not installed |
||||
- **WHEN** a package exists only in sync DB or AUR (not in local DB) |
||||
- **THEN** `PackageInfo.Installed` SHALL be `false` |
||||
|
||||
### Requirement: Mark installed packages as deps, then state packages as explicit |
||||
After package sync completes, the system SHALL mark all installed packages as dependencies, then override the collected state packages to be explicit. This avoids diffing before/after states. |
||||
|
||||
#### Scenario: Mark all installed as deps |
||||
- **WHEN** package sync has completed (non-dry-run) |
||||
- **THEN** the system SHALL run `pacman -D --asdeps` to mark ALL currently installed packages as dependencies |
||||
|
||||
#### Scenario: Override state packages to explicit |
||||
- **WHEN** all installed packages have been marked as deps |
||||
- **THEN** the system SHALL run `pacman -D --asexplicit` on the collected state packages, overriding their dependency status |
||||
|
||||
#### Scenario: Dry-run skips marking |
||||
- **WHEN** operating in dry-run mode |
||||
- **THEN** the system SHALL NOT execute any `pacman -D` marking operations |
||||
@ -0,0 +1,28 @@
|
||||
## ADDED Requirements |
||||
|
||||
### Requirement: Dry-run shows packages to install without making changes |
||||
In dry-run mode, the system SHALL compute what WOULD happen without executing any pacman operations. |
||||
|
||||
#### Scenario: Dry-run lists packages to install |
||||
- **WHEN** dry-run is enabled and packages need to be installed |
||||
- **THEN** the system SHALL populate `Result.ToInstall` with all packages that would be installed (both sync and AUR) |
||||
|
||||
#### Scenario: Dry-run lists packages to remove |
||||
- **WHEN** dry-run is enabled and orphan packages exist |
||||
- **THEN** the system SHALL populate `Result.ToRemove` with the list of orphan packages and `Result.Removed` with the count |
||||
|
||||
#### Scenario: Dry-run skips pacman sync |
||||
- **WHEN** dry-run is enabled |
||||
- **THEN** the system SHALL NOT execute `pacman -Syu` for package installation |
||||
|
||||
#### Scenario: Dry-run skips explicit/deps marking |
||||
- **WHEN** dry-run is enabled |
||||
- **THEN** the system SHALL NOT execute `pacman -D --asdeps` or `pacman -D --asexplicit` |
||||
|
||||
#### Scenario: Dry-run skips orphan cleanup |
||||
- **WHEN** dry-run is enabled |
||||
- **THEN** the system SHALL NOT execute `pacman -Rns` for orphan removal |
||||
|
||||
#### Scenario: Dry-run outputs count summary |
||||
- **WHEN** dry-run is enabled |
||||
- **THEN** the system SHALL still compute and output `Result.Installed` and `Result.Removed` counts as if the operations had run |
||||
@ -0,0 +1,51 @@
|
||||
## 1. Setup |
||||
|
||||
- [x] 1.1 Add `github.com/Jguer/dyalpm` to go.mod |
||||
- [x] 1.2 Run `go mod tidy` to fetch dependencies |
||||
|
||||
## 2. Core Refactoring |
||||
|
||||
- [x] 2.1 Update `PackageInfo` struct to add `Installed bool` field |
||||
- [x] 2.2 Create `Pac` struct with `alpm.Handle` instead of just aurCache |
||||
- [x] 2.3 Implement `NewPac()` that initializes alpm handle and local/sync DBs |
||||
|
||||
## 3. Package Resolution Algorithm |
||||
|
||||
- [x] 3.1 Implement `buildLocalPkgMap()` - iterate localDB.PkgCache() to create lookup map |
||||
- [x] 3.2 Implement `checkSyncDBs()` - iterate each sync DB's PkgCache() to find packages |
||||
- [x] 3.3 Implement `resolvePackages()` - unified algorithm: |
||||
- Step 1: Check local DB for all packages (batch) |
||||
- Step 2: Check sync DBs for remaining packages (batch per repo) |
||||
- Step 3: Batch query AUR for remaining packages |
||||
- Step 4: Return error if any package unfound |
||||
- Step 5: Track installed status from local DB |
||||
|
||||
## 4. Sync and DryRun Integration |
||||
|
||||
- [x] 4.1 Refactor `Sync()` function to use new resolution algorithm |
||||
- [x] 4.2 Refactor `DryRun()` function to use new resolution algorithm |
||||
- [x] 4.3 Preserve AUR batched HTTP calls (existing `fetchAURInfo`) |
||||
- [x] 4.4 Preserve orphan cleanup logic (`CleanupOrphans()`) |
||||
|
||||
## 5. Marking Operations |
||||
|
||||
- [x] 5.1 Keep `MarkExplicit()` for marking state packages |
||||
- [x] 5.2 After sync, run `pacman -D --asdeps` on ALL installed packages (simplifies tracking) |
||||
- [x] 5.3 After deps marking, run `pacman -D --asexplicit` on collected state packages (overrides deps) |
||||
- [x] 5.4 Skip marking operations in dry-run mode |
||||
|
||||
## 6. Cleanup and Output |
||||
|
||||
- [x] 6.1 Remove subprocess-based `ValidatePackage()` implementation |
||||
- [x] 6.2 Remove subprocess-based `GetInstalledPackages()` implementation |
||||
- [x] 6.3 Update output summary to show installed/removed counts |
||||
- [x] 6.4 In dry-run mode, populate `ToInstall` and `ToRemove` lists |
||||
|
||||
## 7. Testing |
||||
|
||||
- [ ] 7.1 Test with packages in local DB only |
||||
- [ ] 7.2 Test with packages in sync DBs only |
||||
- [ ] 7.3 Test with AUR packages |
||||
- [ ] 7.4 Test with missing packages (should error) |
||||
- [ ] 7.5 Test dry-run mode output |
||||
- [ ] 7.6 Test orphan detection and cleanup |
||||
@ -0,0 +1,2 @@
|
||||
schema: spec-driven |
||||
created: 2026-04-14 |
||||
@ -0,0 +1,25 @@
|
||||
## Context |
||||
|
||||
The dyalpm library's `SyncDBs()` returns an empty slice because it doesn't automatically load sync databases from pacman.conf. This causes `resolvePackages()` to skip the sync DB search entirely and fall through to checking the AUR, which fails for official repo packages. |
||||
|
||||
## Goals / Non-Goals |
||||
|
||||
**Goals:** |
||||
- Register sync databases so `resolvePackages()` can find official repo packages |
||||
|
||||
**Non-Goals:** |
||||
- Modify package installation logic |
||||
- Add support for custom repositories |
||||
|
||||
## Decisions |
||||
|
||||
1. **Register each repo manually** - After `handle.SyncDBs()` returns empty, loop through known repos (core, extra, multilib) and call `handle.RegisterSyncDB()` for each. |
||||
|
||||
2. **Use hardcoded repo list** - Arch Linux standard repos are core, extra, multilib. This matches pacman.conf. |
||||
|
||||
3. **Silent failure for missing repos** - If a repo isn't configured in pacman.conf, `RegisterSyncDB` will return a valid but empty DB. Filter by checking if `PkgCache()` has any packages. |
||||
|
||||
## Risks / Trade-offs |
||||
|
||||
- Hardcoded repo names may need updating if Arch adds/removes standard repos → Low risk, rare change |
||||
- Repo registration could fail silently → Mitigated by checking PkgCache count |
||||
@ -0,0 +1,21 @@
|
||||
## Why |
||||
|
||||
The program fails to find packages that exist in official repositories (like `cmake` in `extra`). The dyalpm library's `SyncDBs()` returns an empty list, so the code never searches the sync databases and falls through to checking the AUR, which also doesn't have the package. |
||||
|
||||
## What Changes |
||||
|
||||
- Register sync databases manually after initializing the dyalpm handle |
||||
- Loop through known repos (core, extra, multilib) and call `RegisterSyncDB` for each |
||||
- Handle the case where a repo might not be configured in pacman.conf |
||||
|
||||
## Capabilities |
||||
|
||||
### New Capabilities |
||||
- None - this is a bug fix to existing functionality |
||||
|
||||
### Modified Capabilities |
||||
- None |
||||
|
||||
## Impact |
||||
|
||||
- `pkg/pacman/pacman.go`: Modify `New()` function to register sync DBs after getting them from the handle |
||||
@ -0,0 +1,7 @@
|
||||
## ADDED Requirements |
||||
|
||||
No new requirements - this is a bug fix to existing package resolution functionality. |
||||
|
||||
## MODIFIED Requirements |
||||
|
||||
No modified requirements. |
||||
@ -0,0 +1,10 @@
|
||||
## 1. Implement sync DB registration |
||||
|
||||
- [x] 1.1 Modify Pac struct to include a method for registering sync DBs |
||||
- [x] 1.2 In New(), after getting empty syncDBs from handle, loop through ["core", "extra", "multilib"] and call RegisterSyncDB for each |
||||
- [x] 1.3 Filter out repos that have no packages (not configured in pacman.conf) |
||||
|
||||
## 2. Test the fix |
||||
|
||||
- [x] 2.1 Run `./declpac --dry-run --state <(echo "cmake")` and verify it resolves cmake from extra repo |
||||
- [x] 2.2 Test with other official repo packages (e.g., "git" from extra, "base" from core) |
||||
@ -0,0 +1,2 @@
|
||||
schema: spec-driven |
||||
created: 2026-04-14 |
||||
@ -0,0 +1,141 @@
|
||||
# Design: Refactor pkg Into Modular Packages |
||||
|
||||
## New Structure |
||||
|
||||
``` |
||||
pkg/ |
||||
├── pacman/ # write operations only |
||||
│ └── pacman.go |
||||
├── fetch/ # package resolution (NEW) |
||||
│ └── fetch.go |
||||
├── validation/ # DB freshness (existing, expand) |
||||
│ └── validation.go |
||||
├── output/ # (unchanged) |
||||
│ └── output.go |
||||
├── merge/ # (unchanged) |
||||
│ └── merge.go |
||||
└── input/ # (unchanged) |
||||
└── input.go |
||||
``` |
||||
|
||||
## Package Responsibilities |
||||
|
||||
### pkg/fetch (NEW) |
||||
|
||||
``` |
||||
type Fetcher struct { |
||||
handle dyalpm.Handle |
||||
localDB dyalpm.Database |
||||
syncDBs []dyalpm.Database |
||||
aurCache map[string]AURPackage |
||||
} |
||||
|
||||
func New() *Fetcher |
||||
func (f *Fetcher) Close() error |
||||
func (f *Fetcher) Resolve(packages []string) (map[string]*PackageInfo, error) |
||||
func (f *Fetcher) ListOrphans() ([]string, error) |
||||
``` |
||||
|
||||
Extracted from `pacman.go`: |
||||
- `buildLocalPkgMap()` |
||||
- `checkSyncDBs()` |
||||
- `resolvePackages()` |
||||
- AUR cache (`ensureAURCache()`, `fetchAURInfo()`) |
||||
|
||||
### pkg/pacman (REFACTORED) |
||||
|
||||
``` |
||||
func Sync(packages []string) (*output.Result, error) |
||||
func DryRun(packages []string) (*output.Result, error) |
||||
func MarkAsExplicit(packages []string) error |
||||
func MarkAllAsDeps() error |
||||
func CleanupOrphans() (int, error) |
||||
``` |
||||
|
||||
Write actions only: |
||||
- `Sync()` - calls `Fetcher` for resolution, then pacman commands |
||||
- `DryRun()` - calls `Fetcher.Resolve()` + `ListOrphans()` |
||||
- `MarkAsExplicit()`, `MarkAllAsDeps()` |
||||
- `CleanupOrphans()` - calls `ListOrphans()` then removes |
||||
|
||||
### pkg/validation (REFACTORED) |
||||
|
||||
``` |
||||
func CheckDBFreshness() error |
||||
``` |
||||
|
||||
Keep as-is: checks lock file age, auto-synces if stale. |
||||
|
||||
Remove from `pacman.go`: |
||||
- `IsDBFresh()` - replaced by `CheckDBFreshness()` |
||||
- `SyncDB()` - called by validation when stale |
||||
|
||||
### Orphan Deduplication |
||||
|
||||
```go |
||||
// In fetch/fetch.go |
||||
func (f *Fetcher) ListOrphans() ([]string, error) { |
||||
cmd := exec.Command("pacman", "-Qdtq") |
||||
// ... |
||||
} |
||||
|
||||
// In pacman/pacman.go |
||||
func CleanupOrphans() (int, error) { |
||||
orphans, err := fetcher.ListOrphans() // reuse |
||||
if err != nil || len(orphans) == 0 { |
||||
return 0, nil |
||||
} |
||||
// ... remove |
||||
} |
||||
|
||||
func DryRun(...) (*output.Result, error) { |
||||
orphans, err := fetcher.ListOrphans() // reuse |
||||
// ... |
||||
} |
||||
``` |
||||
|
||||
## Data Structures Move to fetch |
||||
|
||||
```go |
||||
// In fetch/fetch.go |
||||
type PackageInfo struct { |
||||
Name string |
||||
InAUR bool |
||||
Exists bool |
||||
Installed bool |
||||
AURInfo *AURPackage |
||||
syncPkg dyalpm.Package |
||||
} |
||||
|
||||
type AURResponse struct { |
||||
Results []AURPackage `json:"results"` |
||||
} |
||||
|
||||
type AURPackage struct { |
||||
Name string `json:"Name"` |
||||
PackageBase string `json:"PackageBase"` |
||||
Version string `json:"Version"` |
||||
URL string `json:"URL"` |
||||
} |
||||
``` |
||||
|
||||
## Import Changes |
||||
|
||||
`pkg/pacman/fetch.go` will need: |
||||
- `github.com/Jguer/dyalpm` |
||||
- `github.com/Riyyi/declpac/pkg/output` |
||||
|
||||
`pkg/pacman/pacman.go` will need: |
||||
- `github.com/Riyyi/declpac/pkg/fetch` |
||||
- `github.com/Riyyi/declpac/pkg/output` |
||||
|
||||
## Dependencies to Add |
||||
|
||||
- New import: `pkg/fetch` in `pacman` package |
||||
|
||||
## No Changes To |
||||
|
||||
- CLI entry points |
||||
- `output.Result` struct |
||||
- `input.ReadPackages()` |
||||
- `merge.Merge()` |
||||
@ -0,0 +1,30 @@
|
||||
# Proposal: Refactor pkg Into Modular Packages |
||||
|
||||
## Summary |
||||
|
||||
Split monolithic `pkg/pacman/pacman.go` into focused packages: `fetch` (package resolution), `validation` (DB checks), and keep `pacman` for write actions only. Also deduplicate orphan detection logic. |
||||
|
||||
## Motivation |
||||
|
||||
`pacman.go` is 645 lines doing too much: |
||||
- Package resolution (local + sync DBs + AUR) |
||||
- DB freshness checks |
||||
- Pacman write operations (sync, mark, clean) |
||||
- Orphan listing/cleanup |
||||
|
||||
This violates single responsibility. Hard to test, reason about, or reuse. Also has duplication: |
||||
- `validation.CheckDBFreshness()` and `pacman.IsDBFresh()` both check DB freshness |
||||
- `listOrphans()` and `CleanupOrphans()` duplicate orphan detection |
||||
|
||||
## Scope |
||||
|
||||
- Extract package resolution to new `pkg/fetch/` |
||||
- Move DB freshness to `pkg/validation/` (keep `CheckDBFreshness()`) |
||||
- Keep only write actions in `pkg/pacman/` |
||||
- Deduplicate orphan logic: one function for listing, reuse in cleanup and dry-run |
||||
|
||||
## Out of Scope |
||||
|
||||
- No new features |
||||
- No API changes to CLI |
||||
- No changes to `pkg/output/`, `pkg/merge/`, `pkg/input/` |
||||
@ -0,0 +1,15 @@
|
||||
# Scope |
||||
|
||||
## In Scope |
||||
|
||||
- Extract package resolution from pacman.go to pkg/fetch |
||||
- Deduplicate orphan listing |
||||
- Keep pacman write operations in pacman package |
||||
- Maintain existing CLI API |
||||
|
||||
## Out of Scope |
||||
|
||||
- New features |
||||
- New package management backends (e.g., libalpm alternatives) |
||||
- Config file changes |
||||
- State file format changes |
||||
@ -0,0 +1,38 @@
|
||||
# Tasks: Refactor pkg Into Modular Packages |
||||
|
||||
## Phase 1: Create pkg/fetch |
||||
|
||||
- [x] 1.1 Create `pkg/fetch/fetch.go` |
||||
- [x] 1.2 Move `AURResponse`, `AURPackage`, `PackageInfo` structs to fetch |
||||
- [x] 1.3 Move `buildLocalPkgMap()` to fetch as `Fetcher.buildLocalPkgMap()` |
||||
- [x] 1.4 Move `checkSyncDBs()` to fetch as `Fetcher.checkSyncDBs()` |
||||
- [x] 1.5 Move `resolvePackages()` to fetch as `Fetcher.Resolve()` |
||||
- [x] 1.6 Move AUR cache methods (`ensureAURCache`, `fetchAURInfo`) to fetch |
||||
- [x] 1.7 Add `New()` and `Close()` to Fetcher |
||||
- [x] 1.8 Add `ListOrphans()` to Fetcher |
||||
|
||||
## Phase 2: Refactor pkg/pacman |
||||
|
||||
- [x] 2.1 Remove from pacman.go (now in fetch): |
||||
- `buildLocalPkgMap()` |
||||
- `checkSyncDBs()` |
||||
- `resolvePackages()` |
||||
- `ensureAURCache()` |
||||
- `fetchAURInfo()` |
||||
- `AURResponse`, `AURPackage`, `PackageInfo` structs |
||||
- [x] 2.2 Remove `IsDBFresh()` and `SyncDB()` (use validation instead) |
||||
- [x] 2.3 Update imports in pacman.go to include fetch package |
||||
- [x] 2.4 Update `Sync()` to use `fetch.Fetcher` for resolution |
||||
- [x] 2.5 Update `DryRun()` to call `fetcher.ListOrphans()` instead of duplicate call |
||||
- [x] 2.6 Update `CleanupOrphans()` to call `fetcher.ListOrphans()` instead of duplicate call |
||||
|
||||
## Phase 3: Clean Up Validation |
||||
|
||||
- [x] 3.1 Keep `validation.CheckDBFreshness()` as-is |
||||
- [x] 3.2 Remove any remaining DB freshness duplication |
||||
|
||||
## Phase 4: Verify |
||||
|
||||
- [x] 4.1 Run tests (if any exist) |
||||
- [x] 4.2 Build: `go build ./...` |
||||
- [x] 4.3 Verify CLI still works: test dry-run, sync, orphan cleanup |
||||
@ -0,0 +1,345 @@
|
||||
package fetch |
||||
|
||||
import ( |
||||
"encoding/json" |
||||
"fmt" |
||||
"io" |
||||
"net/http" |
||||
"net/url" |
||||
"os" |
||||
"os/exec" |
||||
"strings" |
||||
"time" |
||||
|
||||
"github.com/Jguer/dyalpm" |
||||
) |
||||
|
||||
var ( |
||||
Root = "/" |
||||
LockFile = "/var/lib/pacman/db.lock" |
||||
AURInfoURL = "https://aur.archlinux.org/rpc?v=5&type=info" |
||||
) |
||||
|
||||
type Fetcher struct { |
||||
aurCache map[string]AURPackage |
||||
handle dyalpm.Handle |
||||
localDB dyalpm.Database |
||||
syncDBs []dyalpm.Database |
||||
} |
||||
|
||||
type PackageInfo struct { |
||||
Name string |
||||
InAUR bool |
||||
Exists bool |
||||
Installed bool |
||||
AURInfo *AURPackage |
||||
syncPkg dyalpm.Package |
||||
} |
||||
|
||||
type AURResponse struct { |
||||
Results []AURPackage `json:"results"` |
||||
} |
||||
|
||||
type AURPackage struct { |
||||
Name string `json:"Name"` |
||||
PackageBase string `json:"PackageBase"` |
||||
Version string `json:"Version"` |
||||
URL string `json:"URL"` |
||||
} |
||||
|
||||
func New() (*Fetcher, error) { |
||||
start := time.Now() |
||||
fmt.Fprintf(os.Stderr, "[debug] Fetcher New: starting...\n") |
||||
|
||||
handle, err := dyalpm.Initialize(Root, "/var/lib/pacman") |
||||
if err != nil { |
||||
return nil, fmt.Errorf("failed to initialize alpm: %w", err) |
||||
} |
||||
|
||||
localDB, err := handle.LocalDB() |
||||
if err != nil { |
||||
handle.Release() |
||||
return nil, fmt.Errorf("failed to get local database: %w", err) |
||||
} |
||||
|
||||
syncDBs, err := handle.SyncDBs() |
||||
if err != nil { |
||||
handle.Release() |
||||
return nil, fmt.Errorf("failed to get sync databases: %w", err) |
||||
} |
||||
|
||||
if len(syncDBs) == 0 { |
||||
syncDBs, err = registerSyncDBs(handle) |
||||
if err != nil { |
||||
handle.Release() |
||||
return nil, fmt.Errorf("failed to register sync databases: %w", err) |
||||
} |
||||
} |
||||
|
||||
fmt.Fprintf(os.Stderr, "[debug] Fetcher New: done (%.2fs)\n", time.Since(start).Seconds()) |
||||
return &Fetcher{ |
||||
aurCache: make(map[string]AURPackage), |
||||
handle: handle, |
||||
localDB: localDB, |
||||
syncDBs: syncDBs, |
||||
}, nil |
||||
} |
||||
|
||||
func (f *Fetcher) Close() error { |
||||
if f.handle != nil { |
||||
f.handle.Release() |
||||
} |
||||
return nil |
||||
} |
||||
|
||||
func (f *Fetcher) GetAURPackage(name string) (AURPackage, bool) { |
||||
pkg, ok := f.aurCache[name] |
||||
return pkg, ok |
||||
} |
||||
|
||||
func (f *Fetcher) BuildLocalPkgMap() (map[string]interface{}, error) { |
||||
localPkgs, err := f.buildLocalPkgMap() |
||||
if err != nil { |
||||
return nil, err |
||||
} |
||||
result := make(map[string]interface{}) |
||||
for k, v := range localPkgs { |
||||
result[k] = v |
||||
} |
||||
return result, nil |
||||
} |
||||
|
||||
func registerSyncDBs(handle dyalpm.Handle) ([]dyalpm.Database, error) { |
||||
fmt.Fprintf(os.Stderr, "[debug] registerSyncDBs: starting...\n") |
||||
|
||||
repos := []string{"core", "extra", "multilib"} |
||||
var dbs []dyalpm.Database |
||||
|
||||
for _, repo := range repos { |
||||
db, err := handle.RegisterSyncDB(repo, 0) |
||||
if err != nil { |
||||
continue |
||||
} |
||||
|
||||
count := 0 |
||||
db.PkgCache().ForEach(func(pkg dyalpm.Package) error { |
||||
count++ |
||||
return nil |
||||
}) |
||||
|
||||
if count > 0 { |
||||
dbs = append(dbs, db) |
||||
} |
||||
} |
||||
|
||||
fmt.Fprintf(os.Stderr, "[debug] registerSyncDBs: done (%d dbs)\n", len(dbs)) |
||||
return dbs, nil |
||||
} |
||||
|
||||
func (f *Fetcher) buildLocalPkgMap() (map[string]dyalpm.Package, error) { |
||||
start := time.Now() |
||||
fmt.Fprintf(os.Stderr, "[debug] buildLocalPkgMap: starting...\n") |
||||
|
||||
localPkgs := make(map[string]dyalpm.Package) |
||||
|
||||
err := f.localDB.PkgCache().ForEach(func(pkg dyalpm.Package) error { |
||||
localPkgs[pkg.Name()] = pkg |
||||
return nil |
||||
}) |
||||
|
||||
if err != nil { |
||||
return nil, fmt.Errorf("failed to iterate local package cache: %w", err) |
||||
} |
||||
|
||||
fmt.Fprintf(os.Stderr, "[debug] buildLocalPkgMap: done (%.2fs)\n", time.Since(start).Seconds()) |
||||
return localPkgs, nil |
||||
} |
||||
|
||||
func (f *Fetcher) checkSyncDBs(pkgNames []string) (map[string]dyalpm.Package, error) { |
||||
start := time.Now() |
||||
fmt.Fprintf(os.Stderr, "[debug] checkSyncDBs: starting...\n") |
||||
|
||||
syncPkgs := make(map[string]dyalpm.Package) |
||||
pkgSet := make(map[string]bool) |
||||
for _, name := range pkgNames { |
||||
pkgSet[name] = true |
||||
} |
||||
|
||||
for _, db := range f.syncDBs { |
||||
err := db.PkgCache().ForEach(func(pkg dyalpm.Package) error { |
||||
if pkgSet[pkg.Name()] { |
||||
if _, exists := syncPkgs[pkg.Name()]; !exists { |
||||
syncPkgs[pkg.Name()] = pkg |
||||
} |
||||
} |
||||
return nil |
||||
}) |
||||
if err != nil { |
||||
return nil, fmt.Errorf("failed to iterate sync database %s: %w", db.Name(), err) |
||||
} |
||||
} |
||||
|
||||
fmt.Fprintf(os.Stderr, "[debug] checkSyncDBs: done (%.2fs)\n", time.Since(start).Seconds()) |
||||
return syncPkgs, nil |
||||
} |
||||
|
||||
func (f *Fetcher) Resolve(packages []string) (map[string]*PackageInfo, error) { |
||||
start := time.Now() |
||||
fmt.Fprintf(os.Stderr, "[debug] Resolve: starting...\n") |
||||
|
||||
result := make(map[string]*PackageInfo) |
||||
|
||||
localPkgs, err := f.buildLocalPkgMap() |
||||
if err != nil { |
||||
return nil, err |
||||
} |
||||
fmt.Fprintf(os.Stderr, "[debug] Resolve: local pkgs built (%.2fs)\n", time.Since(start).Seconds()) |
||||
|
||||
var notInLocal []string |
||||
for _, pkg := range packages { |
||||
if localPkg, ok := localPkgs[pkg]; ok { |
||||
result[pkg] = &PackageInfo{ |
||||
Name: pkg, |
||||
Exists: true, |
||||
InAUR: false, |
||||
Installed: true, |
||||
syncPkg: localPkg, |
||||
} |
||||
} else { |
||||
notInLocal = append(notInLocal, pkg) |
||||
} |
||||
} |
||||
|
||||
if len(notInLocal) > 0 { |
||||
syncPkgs, err := f.checkSyncDBs(notInLocal) |
||||
if err != nil { |
||||
return nil, err |
||||
} |
||||
fmt.Fprintf(os.Stderr, "[debug] Resolve: sync db checked (%.2fs)\n", time.Since(start).Seconds()) |
||||
|
||||
var notInSync []string |
||||
for _, pkg := range notInLocal { |
||||
if syncPkg, ok := syncPkgs[pkg]; ok { |
||||
result[pkg] = &PackageInfo{ |
||||
Name: pkg, |
||||
Exists: true, |
||||
InAUR: false, |
||||
Installed: false, |
||||
syncPkg: syncPkg, |
||||
} |
||||
} else { |
||||
notInSync = append(notInSync, pkg) |
||||
} |
||||
} |
||||
|
||||
if len(notInSync) > 0 { |
||||
f.ensureAURCache(notInSync) |
||||
fmt.Fprintf(os.Stderr, "[debug] Resolve: AUR cache ensured (%.2fs)\n", time.Since(start).Seconds()) |
||||
|
||||
var unfound []string |
||||
for _, pkg := range notInSync { |
||||
if aurInfo, ok := f.aurCache[pkg]; ok { |
||||
result[pkg] = &PackageInfo{ |
||||
Name: pkg, |
||||
Exists: true, |
||||
InAUR: true, |
||||
Installed: false, |
||||
AURInfo: &aurInfo, |
||||
} |
||||
} else { |
||||
unfound = append(unfound, pkg) |
||||
} |
||||
} |
||||
if len(unfound) > 0 { |
||||
return nil, fmt.Errorf("package(s) not found: %s", strings.Join(unfound, ", ")) |
||||
} |
||||
} |
||||
} |
||||
|
||||
fmt.Fprintf(os.Stderr, "[debug] Resolve: done (%.2fs)\n", time.Since(start).Seconds()) |
||||
return result, nil |
||||
} |
||||
|
||||
func (f *Fetcher) ensureAURCache(packages []string) { |
||||
start := time.Now() |
||||
fmt.Fprintf(os.Stderr, "[debug] ensureAURCache: starting...\n") |
||||
|
||||
if len(packages) == 0 { |
||||
return |
||||
} |
||||
|
||||
var uncached []string |
||||
for _, pkg := range packages { |
||||
if _, ok := f.aurCache[pkg]; !ok { |
||||
uncached = append(uncached, pkg) |
||||
} |
||||
} |
||||
|
||||
if len(uncached) == 0 { |
||||
fmt.Fprintf(os.Stderr, "[debug] ensureAURCache: done (%.2fs)\n", time.Since(start).Seconds()) |
||||
return |
||||
} |
||||
|
||||
f.fetchAURInfo(uncached) |
||||
fmt.Fprintf(os.Stderr, "[debug] ensureAURCache: done (%.2fs)\n", time.Since(start).Seconds()) |
||||
} |
||||
|
||||
func (f *Fetcher) fetchAURInfo(packages []string) map[string]AURPackage { |
||||
start := time.Now() |
||||
fmt.Fprintf(os.Stderr, "[debug] fetchAURInfo: starting...\n") |
||||
|
||||
result := make(map[string]AURPackage) |
||||
|
||||
if len(packages) == 0 { |
||||
return result |
||||
} |
||||
|
||||
v := url.Values{} |
||||
for _, pkg := range packages { |
||||
v.Add("arg[]", pkg) |
||||
} |
||||
|
||||
resp, err := http.Get(AURInfoURL + "&" + v.Encode()) |
||||
if err != nil { |
||||
return result |
||||
} |
||||
defer resp.Body.Close() |
||||
|
||||
body, err := io.ReadAll(resp.Body) |
||||
if err != nil { |
||||
return result |
||||
} |
||||
|
||||
var aurResp AURResponse |
||||
if err := json.Unmarshal(body, &aurResp); err != nil { |
||||
return result |
||||
} |
||||
|
||||
for _, r := range aurResp.Results { |
||||
f.aurCache[r.Name] = r |
||||
result[r.Name] = r |
||||
} |
||||
|
||||
fmt.Fprintf(os.Stderr, "[debug] fetchAURInfo: done (%.2fs)\n", time.Since(start).Seconds()) |
||||
return result |
||||
} |
||||
|
||||
func (f *Fetcher) ListOrphans() ([]string, error) { |
||||
start := time.Now() |
||||
fmt.Fprintf(os.Stderr, "[debug] ListOrphans: starting...\n") |
||||
|
||||
cmd := exec.Command("pacman", "-Qdtq") |
||||
orphans, err := cmd.Output() |
||||
if err != nil { |
||||
return nil, nil |
||||
} |
||||
|
||||
list := strings.TrimSpace(string(orphans)) |
||||
if list == "" { |
||||
fmt.Fprintf(os.Stderr, "[debug] ListOrphans: done (%.2fs)\n", time.Since(start).Seconds()) |
||||
return nil, nil |
||||
} |
||||
|
||||
fmt.Fprintf(os.Stderr, "[debug] ListOrphans: done (%.2fs)\n", time.Since(start).Seconds()) |
||||
return strings.Split(list, "\n"), nil |
||||
} |
||||
Loading…
Reference in new issue