Add specs for dyalpm package query and batch explicit marking

openspec/changes/use-dyalpm-and-batch-explicit/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-04-13

openspec/changes/use-dyalpm-and-batch-explicit/design.md

@@ -0,0 +1,67 @@
+## Context
+
+Current implementation in `pkg/pacman/pacman.go` spawns shell processes for pacman queries:
+
+1. **`ValidatePackage`**: Calls `pacman -Qip` then `pacman -Sip` (2 processes per package) to check if package exists in sync databases
+2. **`MarkExplicit`**: Calls `pacman -D --explicit <pkg>` individually per package (N processes for N packages)
+3. **AUR**: Called individually per package not found in pacman databases
+
+Performance issues scale with package count. No caching of package queries across calls.
+
+## Goals / Non-Goals
+
+**Goals:**
+- Use Jguer/dyalpm library to query pacman databases without spawning processes
+- Add in-memory cache for ALL package query results (pacman + AUR), valid for entire job duration
+- Batch `pacman -D --explicit` calls to single process for multiple packages
+- Batch AUR HTTP queries to single request for all packages not found in pacman
+
+**Non-Goals:**
+- Refactor AUR handling (already uses HTTP API - will batch it)
+- Add persistent cache (only job-duration in-memory)
+- Change other pacman operations (sync, cleanup)
+
+## Decisions
+
+1. **dyalpm over go-alpm**: dyalpm uses purego (no cgo), cleaner cross-compilation
+   - Alternative: go-alpm (cgo-based) - rejected for compilation complexity
+
+2. **Unified cache in Pac struct**: Single cache map replaces separate aurCache
+   - Alternative: keep separate caches - rejected, unnecessary complexity
+   - Cache key: package name, value: PackageInfo struct
+
+3. **Batch MarkExplicit**: Accept `[]string` packages, pass all to single `pacman -D --explicit` call
+   - Note: pacman -D accepts multiple packages in single call
+
+4. **Batch query strategy**:
+   - Query all packages against dyalpm local DB → returns found[]
+   - Query not-found against dyalpm sync DBs → returns found[]
+   - Query remaining not-found against AUR HTTP (single batched request)
+   - Current: pacman -Qip → pacman -Sip → AUR (per-package)
+   - New: Batch dyalpm local → Batch dyalpm sync → Batch AUR
+
+5. **Fallback for dyalpm unavailable**: If dyalpm init fails, fall back to:
+   - pacman -Qip for all packages (single process, capture all)
+   - pacman -Sip for remaining (single process, capture all)
+   - AUR HTTP batch for remaining (already batched)
+
+## Risks / Trade-offs
+
+- **Risk**: dyalpm requires libalpm.so.15 on system
+  - Mitigation: Check at runtime, fallback to process spawn if missing
+
+- **Risk**: Cache invalidation edge cases (e.g., package installed during job)
+  - Mitigation: Acceptable for declpac use case; user runs sync after config changes
+
+- **Risk**: AUR API batch size limits
+  - Mitigation: Chunk large batches if AUR has limits (TBD in implementation)
+
+## Migration Plan
+
+1. Add dyalpm dependency to go.mod
+2. Refactor Pac struct: replace aurCache with unified pkgCache map
+3. Change ValidatePackage to ValidatePackages (slice input, batch processing)
+4. Update MarkExplicit to accept slice
+5. Batch AUR HTTP calls in ensureAURCache
+6. Update call sites in Sync(), categorizePackages, DryRun()
+7. Test with existing test suite

openspec/changes/use-dyalpm-and-batch-explicit/proposal.md

@@ -0,0 +1,28 @@
+## Why
+
+Current `ValidatePackage` spawns multiple `pacman` processes per package (-Qip, -Sip calls), causing performance issues when checking many packages. Additionally, `MarkExplicit` is called individually for each package, spawning a separate process per package. Finally, AUR HTTP calls are made per-package. Using the Jguer/dyalpm Go library eliminates process spawning overhead and enables efficient batch operations throughout.
+
+## What Changes
+
+- Replace shell-out to `pacman -Qip`/`pacman -Sip` in `ValidatePackage` with Jguer/dyalpm library calls
+- Add unified in-memory package cache to `Pac` struct (replaces existing aurCache), persisting for job duration
+- Replace individual `pacman -D --explicit` calls with single batch call
+- Batch AUR HTTP queries into single request for all packages not found in pacman databases
+- Add Jguer/dyalpm dependency
+
+## Capabilities
+
+### New Capabilities
+
+- **dyalpm-package-query**: Use dyalpm library for querying pacman package databases in batch, falling back to AUR HTTP batch for remaining packages. Results cached for job duration.
+- **batch-explicit-mark**: Batch multiple packages into single `pacman -D --explicit` call instead of per-package process spawn.
+
+### Modified Capabilities
+
+None.
+
+## Impact
+
+- **pkg/pacman/pacman.go**: Refactor `ValidatePackage` to accept slice, replace aurCache with unified pkgCache, refactor `MarkExplicit` to accept slice
+- **go.mod**: Add Jguer/dyalpm import
+- **Performance**: Reduced process spawns from O(n*2 + n AUR) to O(1) for ValidatePackages, O(n) to O(1) for MarkExplicit

openspec/changes/use-dyalpm-and-batch-explicit/specs/batch-explicit-mark/spec.md

@@ -0,0 +1,16 @@
+## ADDED Requirements
+
+### Requirement: MarkExplicit accepts multiple packages
+The MarkExplicit method SHALL accept a slice of package names and mark all of them as explicitly installed using a single pacman -D call.
+
+#### Scenario: Single package marked explicit
+- **WHEN** MarkExplicit is called with one package name
+- **THEN** pacman -D --explicit is called once with that package
+
+#### Scenario: Multiple packages marked explicit
+- **WHEN** MarkExplicit is called with multiple package names (e.g., ["pkg1", "pkg2"])
+- **THEN** pacman -D --explicit is called once with all packages as arguments
+
+#### Scenario: Empty package slice
+- **WHEN** MarkExplicit is called with an empty slice
+- **THEN** no pacman call is made, method returns nil immediately

openspec/changes/use-dyalpm-and-batch-explicit/specs/dyalpm-package-query/spec.md

@@ -0,0 +1,47 @@
+## ADDED Requirements
+
+### Requirement: ValidatePackages uses dyalpm library with batch AUR fallback
+The ValidatePackages method SHALL accept a slice of package names, query them in batch using dyalpm, then batch query AUR for any not found in pacman databases.
+
+#### Scenario: All packages found in local database
+- **WHEN** ValidatePackages is called with packages that all exist in local pacman database
+- **THEN** returns a slice of PackageInfo with Exists=true, InAUR=false for each
+
+#### Scenario: Some packages found in sync database
+- **WHEN** ValidatePackages is called with packages where some are only in sync databases
+- **THEN** returns PackageInfo with Exists=true, InAUR=false for found packages
+
+#### Scenario: Some packages found in AUR
+- **WHEN** ValidatePackages is called and some packages are not in pacman databases
+- **THEN** those packages are batched to single AUR HTTP request, returns PackageInfo with InAUR=true
+
+#### Scenario: Packages not found anywhere
+- **WHEN** ValidatePackages is called and some packages not found in pacman or AUR
+- **THEN** returns PackageInfo with Exists=false, InAUR=false for those packages
+
+### Requirement: Unified package cache
+The Pac struct SHALL maintain a single cache map that stores all package query results (both pacman and AUR), replacing the separate aurCache.
+
+#### Scenario: Package already cached
+- **WHEN** ValidatePackages is called for a package already queried in current job
+- **THEN** cached result returned without any dyalpm or AUR calls
+
+#### Scenario: New Pac instance
+- **WHEN** New() creates a new Pac instance
+- **THEN** cache is empty (fresh job)
+
+### Requirement: Single batch AUR call
+The AUR HTTP API SHALL be called once with all package names not found in pacman databases.
+
+#### Scenario: Multiple packages not in pacman
+- **WHEN** 3 packages not found in local or sync databases
+- **THEN** single AUR HTTP request with all 3 in arg[] params
+- **AND** results cached for all 3
+
+### Requirement: Fallback to process spawn if dyalpm unavailable
+If dyalpm library is not available at runtime, ValidatePackages SHALL fall back to spawning pacman -Qip/-Sip processes in batch.
+
+#### Scenario: dyalpm unavailable
+- **WHEN** dyalpm initialization fails
+- **THEN** uses pacman -Qip for local, -Sip for sync, AUR HTTP batch as fallback
+- **AND** behavior is identical to using dyalpm

openspec/changes/use-dyalpm-and-batch-explicit/tasks.md

@@ -0,0 +1,36 @@
+## 1. Dependencies
+
+- [ ] 1.1 Add github.com/Jguer/dyalpm to go.mod
+- [ ] 1.2 Run go mod tidy to resolve dependencies
+
+## 2. Pac Struct Updates
+
+- [ ] 2.1 Add dyalpm handle field to Pac struct
+- [ ] 2.2 Add package info cache field (map[string]PackageInfo) to Pac struct
+- [ ] 2.3 Update New() to initialize dyalpm handle
+- [ ] 2.4 Update Close() to release dyalpm handle
+
+## 3. ValidatePackage Implementation
+
+- [ ] 3.1 Implement ValidatePackage using dyalpm LocalDB().Pkg()
+- [ ] 3.2 Query sync databases if not found locally
+- [ ] 3.3 Add caching logic to store/query results
+- [ ] 3.4 Keep AUR fallback for packages not in pacman repos
+
+## 4. MarkExplicit Updates
+
+- [ ] 4.1 Change MarkExplicit signature to accept []string (slice of packages)
+- [ ] 4.2 Implement single pacman -D --explicit call with all packages
+- [ ] 4.3 Handle empty slice case (return nil immediately)
+
+## 5. Call Site Updates
+
+- [ ] 5.1 Update Sync() to pass packages slice to MarkExplicit
+- [ ] 5.2 Ensure categorizePackages works with new ValidatePackage
+
+## 6. Testing
+
+- [ ] 6.1 Run existing tests to verify no regressions
+- [ ] 6.2 Test ValidatePackage with local, sync, and AUR packages
+- [ ] 6.3 Test MarkExplicit with single and multiple packages
+- [ ] 6.4 Verify cache behavior (second call returns cached result)