diff --git a/openspec/changes/use-dyalpm-and-batch-explicit/.openspec.yaml b/openspec/changes/use-dyalpm-and-batch-explicit/.openspec.yaml new file mode 100644 index 0000000..e14f322 --- /dev/null +++ b/openspec/changes/use-dyalpm-and-batch-explicit/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-04-13 diff --git a/openspec/changes/use-dyalpm-and-batch-explicit/design.md b/openspec/changes/use-dyalpm-and-batch-explicit/design.md new file mode 100644 index 0000000..a5dc658 --- /dev/null +++ b/openspec/changes/use-dyalpm-and-batch-explicit/design.md @@ -0,0 +1,67 @@ +## Context + +Current implementation in `pkg/pacman/pacman.go` spawns shell processes for pacman queries: + +1. **`ValidatePackage`**: Calls `pacman -Qip` then `pacman -Sip` (2 processes per package) to check if package exists in sync databases +2. **`MarkExplicit`**: Calls `pacman -D --explicit ` individually per package (N processes for N packages) +3. **AUR**: Called individually per package not found in pacman databases + +Performance issues scale with package count. No caching of package queries across calls. + +## Goals / Non-Goals + +**Goals:** +- Use Jguer/dyalpm library to query pacman databases without spawning processes +- Add in-memory cache for ALL package query results (pacman + AUR), valid for entire job duration +- Batch `pacman -D --explicit` calls to single process for multiple packages +- Batch AUR HTTP queries to single request for all packages not found in pacman + +**Non-Goals:** +- Refactor AUR handling (already uses HTTP API - will batch it) +- Add persistent cache (only job-duration in-memory) +- Change other pacman operations (sync, cleanup) + +## Decisions + +1. **dyalpm over go-alpm**: dyalpm uses purego (no cgo), cleaner cross-compilation + - Alternative: go-alpm (cgo-based) - rejected for compilation complexity + +2. **Unified cache in Pac struct**: Single cache map replaces separate aurCache + - Alternative: keep separate caches - rejected, unnecessary complexity + - Cache key: package name, value: PackageInfo struct + +3. **Batch MarkExplicit**: Accept `[]string` packages, pass all to single `pacman -D --explicit` call + - Note: pacman -D accepts multiple packages in single call + +4. **Batch query strategy**: + - Query all packages against dyalpm local DB → returns found[] + - Query not-found against dyalpm sync DBs → returns found[] + - Query remaining not-found against AUR HTTP (single batched request) + - Current: pacman -Qip → pacman -Sip → AUR (per-package) + - New: Batch dyalpm local → Batch dyalpm sync → Batch AUR + +5. **Fallback for dyalpm unavailable**: If dyalpm init fails, fall back to: + - pacman -Qip for all packages (single process, capture all) + - pacman -Sip for remaining (single process, capture all) + - AUR HTTP batch for remaining (already batched) + +## Risks / Trade-offs + +- **Risk**: dyalpm requires libalpm.so.15 on system + - Mitigation: Check at runtime, fallback to process spawn if missing + +- **Risk**: Cache invalidation edge cases (e.g., package installed during job) + - Mitigation: Acceptable for declpac use case; user runs sync after config changes + +- **Risk**: AUR API batch size limits + - Mitigation: Chunk large batches if AUR has limits (TBD in implementation) + +## Migration Plan + +1. Add dyalpm dependency to go.mod +2. Refactor Pac struct: replace aurCache with unified pkgCache map +3. Change ValidatePackage to ValidatePackages (slice input, batch processing) +4. Update MarkExplicit to accept slice +5. Batch AUR HTTP calls in ensureAURCache +6. Update call sites in Sync(), categorizePackages, DryRun() +7. Test with existing test suite \ No newline at end of file diff --git a/openspec/changes/use-dyalpm-and-batch-explicit/proposal.md b/openspec/changes/use-dyalpm-and-batch-explicit/proposal.md new file mode 100644 index 0000000..02ca8ae --- /dev/null +++ b/openspec/changes/use-dyalpm-and-batch-explicit/proposal.md @@ -0,0 +1,28 @@ +## Why + +Current `ValidatePackage` spawns multiple `pacman` processes per package (-Qip, -Sip calls), causing performance issues when checking many packages. Additionally, `MarkExplicit` is called individually for each package, spawning a separate process per package. Finally, AUR HTTP calls are made per-package. Using the Jguer/dyalpm Go library eliminates process spawning overhead and enables efficient batch operations throughout. + +## What Changes + +- Replace shell-out to `pacman -Qip`/`pacman -Sip` in `ValidatePackage` with Jguer/dyalpm library calls +- Add unified in-memory package cache to `Pac` struct (replaces existing aurCache), persisting for job duration +- Replace individual `pacman -D --explicit` calls with single batch call +- Batch AUR HTTP queries into single request for all packages not found in pacman databases +- Add Jguer/dyalpm dependency + +## Capabilities + +### New Capabilities + +- **dyalpm-package-query**: Use dyalpm library for querying pacman package databases in batch, falling back to AUR HTTP batch for remaining packages. Results cached for job duration. +- **batch-explicit-mark**: Batch multiple packages into single `pacman -D --explicit` call instead of per-package process spawn. + +### Modified Capabilities + +None. + +## Impact + +- **pkg/pacman/pacman.go**: Refactor `ValidatePackage` to accept slice, replace aurCache with unified pkgCache, refactor `MarkExplicit` to accept slice +- **go.mod**: Add Jguer/dyalpm import +- **Performance**: Reduced process spawns from O(n*2 + n AUR) to O(1) for ValidatePackages, O(n) to O(1) for MarkExplicit \ No newline at end of file diff --git a/openspec/changes/use-dyalpm-and-batch-explicit/specs/batch-explicit-mark/spec.md b/openspec/changes/use-dyalpm-and-batch-explicit/specs/batch-explicit-mark/spec.md new file mode 100644 index 0000000..2d94610 --- /dev/null +++ b/openspec/changes/use-dyalpm-and-batch-explicit/specs/batch-explicit-mark/spec.md @@ -0,0 +1,16 @@ +## ADDED Requirements + +### Requirement: MarkExplicit accepts multiple packages +The MarkExplicit method SHALL accept a slice of package names and mark all of them as explicitly installed using a single pacman -D call. + +#### Scenario: Single package marked explicit +- **WHEN** MarkExplicit is called with one package name +- **THEN** pacman -D --explicit is called once with that package + +#### Scenario: Multiple packages marked explicit +- **WHEN** MarkExplicit is called with multiple package names (e.g., ["pkg1", "pkg2"]) +- **THEN** pacman -D --explicit is called once with all packages as arguments + +#### Scenario: Empty package slice +- **WHEN** MarkExplicit is called with an empty slice +- **THEN** no pacman call is made, method returns nil immediately \ No newline at end of file diff --git a/openspec/changes/use-dyalpm-and-batch-explicit/specs/dyalpm-package-query/spec.md b/openspec/changes/use-dyalpm-and-batch-explicit/specs/dyalpm-package-query/spec.md new file mode 100644 index 0000000..4298f45 --- /dev/null +++ b/openspec/changes/use-dyalpm-and-batch-explicit/specs/dyalpm-package-query/spec.md @@ -0,0 +1,47 @@ +## ADDED Requirements + +### Requirement: ValidatePackages uses dyalpm library with batch AUR fallback +The ValidatePackages method SHALL accept a slice of package names, query them in batch using dyalpm, then batch query AUR for any not found in pacman databases. + +#### Scenario: All packages found in local database +- **WHEN** ValidatePackages is called with packages that all exist in local pacman database +- **THEN** returns a slice of PackageInfo with Exists=true, InAUR=false for each + +#### Scenario: Some packages found in sync database +- **WHEN** ValidatePackages is called with packages where some are only in sync databases +- **THEN** returns PackageInfo with Exists=true, InAUR=false for found packages + +#### Scenario: Some packages found in AUR +- **WHEN** ValidatePackages is called and some packages are not in pacman databases +- **THEN** those packages are batched to single AUR HTTP request, returns PackageInfo with InAUR=true + +#### Scenario: Packages not found anywhere +- **WHEN** ValidatePackages is called and some packages not found in pacman or AUR +- **THEN** returns PackageInfo with Exists=false, InAUR=false for those packages + +### Requirement: Unified package cache +The Pac struct SHALL maintain a single cache map that stores all package query results (both pacman and AUR), replacing the separate aurCache. + +#### Scenario: Package already cached +- **WHEN** ValidatePackages is called for a package already queried in current job +- **THEN** cached result returned without any dyalpm or AUR calls + +#### Scenario: New Pac instance +- **WHEN** New() creates a new Pac instance +- **THEN** cache is empty (fresh job) + +### Requirement: Single batch AUR call +The AUR HTTP API SHALL be called once with all package names not found in pacman databases. + +#### Scenario: Multiple packages not in pacman +- **WHEN** 3 packages not found in local or sync databases +- **THEN** single AUR HTTP request with all 3 in arg[] params +- **AND** results cached for all 3 + +### Requirement: Fallback to process spawn if dyalpm unavailable +If dyalpm library is not available at runtime, ValidatePackages SHALL fall back to spawning pacman -Qip/-Sip processes in batch. + +#### Scenario: dyalpm unavailable +- **WHEN** dyalpm initialization fails +- **THEN** uses pacman -Qip for local, -Sip for sync, AUR HTTP batch as fallback +- **AND** behavior is identical to using dyalpm \ No newline at end of file diff --git a/openspec/changes/use-dyalpm-and-batch-explicit/tasks.md b/openspec/changes/use-dyalpm-and-batch-explicit/tasks.md new file mode 100644 index 0000000..a08bd26 --- /dev/null +++ b/openspec/changes/use-dyalpm-and-batch-explicit/tasks.md @@ -0,0 +1,36 @@ +## 1. Dependencies + +- [ ] 1.1 Add github.com/Jguer/dyalpm to go.mod +- [ ] 1.2 Run go mod tidy to resolve dependencies + +## 2. Pac Struct Updates + +- [ ] 2.1 Add dyalpm handle field to Pac struct +- [ ] 2.2 Add package info cache field (map[string]PackageInfo) to Pac struct +- [ ] 2.3 Update New() to initialize dyalpm handle +- [ ] 2.4 Update Close() to release dyalpm handle + +## 3. ValidatePackage Implementation + +- [ ] 3.1 Implement ValidatePackage using dyalpm LocalDB().Pkg() +- [ ] 3.2 Query sync databases if not found locally +- [ ] 3.3 Add caching logic to store/query results +- [ ] 3.4 Keep AUR fallback for packages not in pacman repos + +## 4. MarkExplicit Updates + +- [ ] 4.1 Change MarkExplicit signature to accept []string (slice of packages) +- [ ] 4.2 Implement single pacman -D --explicit call with all packages +- [ ] 4.3 Handle empty slice case (return nil immediately) + +## 5. Call Site Updates + +- [ ] 5.1 Update Sync() to pass packages slice to MarkExplicit +- [ ] 5.2 Ensure categorizePackages works with new ValidatePackage + +## 6. Testing + +- [ ] 6.1 Run existing tests to verify no regressions +- [ ] 6.2 Test ValidatePackage with local, sync, and AUR packages +- [ ] 6.3 Test MarkExplicit with single and multiple packages +- [ ] 6.4 Verify cache behavior (second call returns cached result)