All extract tests pass.
>go test -run TestParseDep5 -v 2>&1; go test -run TestDep5 -v 2>&1>go test -run "TestParseDep5|TestDep5" -vAll dep5 tests pass.
>go mod tidy>go test -run "TestParseReuseTOML|TestAnnotation|TestReuseTOML" -vAll TOML tests pass.
>go test -run "TestIsIgnored|TestIsCovered|TestCoveredFiles" -vAll covered file tests pass.
Now let me create the test fixtures and project tests.
>go test -run "TestOpenProject|TestProject" -vAll project tests pass (the reuse-example test is correctly skipped since we haven't added the submodule yet).
Now let me add the submodule, fake_repository testdata, CI workflows, and README.
>git submodule add https://github.com/fsfe/reuse-example.git testdata/reuse-example>ls /Users/andrew/code/git-pkgs/reuse/testdata/reuse-example/Let me check what's in this repo to understand the structure for the conformance test.
>find /Users/andrew/code/git-pkgs/reuse/testdata/reuse-example -not -path '*/.git/*' -type f | sortGood, the reuse-example has headers in source files and .license sidecars for images. No REUSE.toml or dep5, just header-only. Let me now run the conformance test to see if it passes.
>go test -run TestProject_ReuseExample -vThe conformance test passes against the real reuse-example repo. Now let me create the fake_repository testdata and infrastructure files.
>mkdir -p /Users/andrew/code/git-pkgs/reuse/testdata/fake_repository/{LICENSES,src,.reuse} /Users/andrew/code/git-pkgs/reuse/testdata/fake_repository/assetsImplement the following plan:
Parsing library for the REUSE specification v3.3 at github.com/git-pkgs/reuse. Extracts SPDX license and copyright information from files, REUSE.toml, .reuse/dep5, and .license sidecars. No CLI, no linting output, just parsing.
git-pkgs needs to extract licensing information from repositories. The REUSE spec (fsfe.org) standardises where and how license/copyright info lives in a project. The Python reference implementation (fsfe/reuse-tool) is GPL-licensed and has complex dependencies. A focused Go parsing library lets git-pkgs consume REUSE data without shelling out to Python.
Add fsfe/reuse-example as a git submodule at testdata/reuse-example for conformance testing. It has a compliant main branch with LICENSES/, SPDX headers in source, .license sidecars for images.
Also create testdata/fake_repository/ modelled on the reuse-tool's test fixtures, with various license files, source files with headers, and edge cases.
github.com/BurntSushi/toml for REUSE.toml parsing (Go stdlib has no TOML parser)All under /Users/andrew/code/git-pkgs/reuse/.
reuse.go - Package docs and top-level typesCore types that other files use:
type ReuseInfo struct { LicenseExpressions []string CopyrightNotices []string Contributors []string SourcePath string // where this info came from SourceType SourceType // file-header, dot-license, reuse-toml, dep5}
type SourceType int // FileHeader, DotLicense, ReuseToml, Dep5
type PrecedenceType int // Closest, Aggregate, Overrideextract.go - SPDX tag extraction from file contentsPort of Python's extract.py. The core parsing engine.
ExtractReuseInfo(text string) ReuseInfo - find SPDX-License-Identifier, SPDX-FileCopyrightText, SPDX-FileContributor tags in textExtractFromFile(path string) (ReuseInfo, error) - read a file and extractFilterIgnoreBlocks(text string) string - strip REUSE-IgnoreStart/End regionsextract_test.gotoml.go - REUSE.toml parsingPort of Python's global_licensing.py (ReuseTOML parts).
type ReuseTOML struct { Version int Annotations []Annotation Source string}
type Annotation struct { Paths []string Precedence PrecedenceType Copyrights []string Licenses []string}ParseReuseTOML(content string) (*ReuseTOML, error)ParseReuseTOMLFile(path string) (*ReuseTOML, error)(a *Annotation) Matches(path string) bool - glob matching with * and ** support(t *ReuseTOML) ReuseInfoOf(path string) (ReuseInfo, PrecedenceType, bool) - find matching annotation for a pathtoml_test.godep5.go - .reuse/dep5 parsingMinimal Debian copyright format 1.0 parser (no external dep).
type Dep5 struct { Header Dep5Header Files []Dep5FilesParagraph}
type Dep5FilesParagraph struct { Patterns []string Copyright string License string}ParseDep5(content string) (*Dep5, error)ParseDep5File(path string) (*Dep5, error)(d *Dep5) ReuseInfoOf(path string) (ReuseInfo, bool) - find matching paragraph for a pathdep5_test.gocovered.go - Covered file logicDetermines which files need licensing info and which are excluded per the spec.
IsCoveredFile(path string) bool - checks against exclusion patternsIsIgnoredDir(name string) bool - .git, .hg, LICENSES, .reuseIsIgnoredFile(name string) bool - LICENSE*, COPYING*, *.license, REUSE.toml, .spdxCoveredFiles(root string) ([]string, error) - walk directory returning covered filescovered_test.goproject.go - Project-level parsingTies everything together. Given a project root, find and parse all licensing info.
type Project struct { Root string ReuseTOML *ReuseTOML // nil if not present Dep5 *Dep5 // nil if not present LicenseFiles []string // paths in LICENSES/}OpenProject(root string) (*Project, error) - discover REUSE.toml or dep5, scan LICENSES/(p *Project) ReuseInfoOf(path string) (ReuseInfo, error) - resolve all sources with precedence:
(p *Project) AllReuseInfo() (map[string]ReuseInfo, error) - walk all covered filesproject_test.goglob.go - REUSE.toml glob matchingCustom glob implementation matching the REUSE.toml spec:
* matches everything except /
** matches everything including /
\\* is a literal asterisk
Forward slashes only
GlobMatch(pattern, path string) bool
glob_test.go*.go matches foo.go but not dir/foo.go**/*.go matches dir/foo.go and dir/sub/foo.gosrc/** matches everything under src/go.mod - module github.com/git-pkgs/reuse, go 1.25.6, deps on BurntSushi/toml and git-pkgs/spdxLICENSE - MIT, copied from versREADME.md - usage examples.gitignore - *.test, *.prof.gitmodules - testdata/reuse-example submodule.github/workflows/ci.yml - matching other modules (with submodules: true).github/workflows/zizmor.yml - matching other modulesreuse.go - typesglob.go + tests - glob matching (no deps on other files)extract.go + tests - SPDX tag extraction (no deps on other files)dep5.go + tests - dep5 parsingtoml.go + tests - REUSE.toml parsing (depends on glob.go)covered.go + tests - covered file logicproject.go + tests - ties it all togethergo test -v -race ./... - all tests passgo vet ./... - cleanIf you need specific details from before exiting plan mode (like exact code snippets, error messages, or content you generated), read the full transcript at: /Users/andrew/.claude/projects/-Users-andrew-code-git-pkgs/ced24506-3bc7-4de2-84fd-76adc53e1512.jsonl
Now add a test using the fake_repository.
>go test -run TestProject_FakeRepository -vNow let me create the CI workflow and README.
>mkdir -p /Users/andrew/code/git-pkgs/reuse/.github/workflowsNow the README.
Now let's run the full test suite and go vet.
>go test -v -race ./...All 64 tests pass with the race detector. Let me also run go vet.
>go vet ./...Clean. Let me also clean up the unused function flagged by the linter.
Now check if filepath is still needed in dep5.go after removing that function.
Let me start by exploring sibling modules for patterns and the REUSE spec.
>go test -v -race ./... && go vet ./...64 tests pass, go vet clean.
Everything is done. Here's a summary of what was built:
Files created (all under /Users/andrew/code/git-pkgs/reuse/):
| File | Purpose |
|---|---|
reuse.go | Core types: ReuseInfo, SourceType, PrecedenceType |
glob.go + glob_test.go | REUSE.toml glob matching (*, **, escapes) |
extract.go + extract_test.go | SPDX tag extraction from file contents (license, copyright, contributor, ignore blocks, snippets) |
dep5.go + dep5_test.go | Minimal Debian copyright format parser for .reuse/dep5 |
toml.go + toml_test.go | REUSE.toml parsing with annotations, precedence, string-or-array fields |
covered.go + covered_test.go | Covered file logic (ignored dirs/files, symlinks, zero-byte) |
project.go + project_test.go | Project-level parsing tying everything together with precedence resolution |
go.mod / go.sum | Module github.com/git-pkgs/reuse, Go 1.25.6, dep on BurntSushi/toml |
LICENSE | MIT (from SPDX) |
README.md | Usage examples |
.gitignore | *.test, *.prof |
.gitmodules | testdata/reuse-example submodule |
.github/workflows/ci.yml | CI matching sibling modules (with submodules: true) |
.github/dependabot.yml | Weekly updates for gomod + github-actions |
testdata/reuse-example/ | fsfe/reuse-example submodule for conformance testing |
testdata/fake_repository/ | Edge case fixtures (dual licensing, ignore blocks, sidecars, no-header files) |
Test results: 64 tests pass with -race, go vet clean. The conformance test against the real fsfe/reuse-example repo confirms all covered files get both license and copyright extracted correctly.
Ready for you to review. Want me to commit?
can we organize the modules into a folder rather than one big flat folder
Good idea. Let me look at what we have and figure out a sensible split.
>ls /Users/andrew/code/git-pkgs/reuse/*.goThe sibling modules that have subpackages use an internal/ pattern with a core package for shared types and a registration system. But the reuse module doesn't need a registration system - it has distinct parsers (extract, dep5, toml) and a project orchestrator.
For reuse, the natural split by concern would be:
Root package: types, Project (the main entry point), covered file logic
extract/ - SPDX tag extraction from file contents
dep5/ - dep5 parsing
toml/ - REUSE.toml parsing (but "toml" conflicts with the BurntSushi/toml import name)
glob/ - glob matching
Looking at how the sibling packages structure things with internal/ subpackages, I realize the user is asking for public subpackages organized by concern rather than hidden internal ones. So I'm thinking through the public API: reuse/extract, reuse/dep5, reuse/toml (though that naming conflicts with the external toml library, maybe reusetoml instead), and reuse/glob for matching - though glob might be too small to warrant its own package.
For the actual structure, the root package would expose the main types like ReuseInfo, SourceType, PrecedenceType, and Project, while the subpackages handle specific concerns: extract for SPDX extraction, dep5 for parsing DEP5 files, and toml for REUSE.toml configuration.
The glob matching logic is only used by the TOML parser, so it makes sense to keep that internal to that package. And covered.go stays in the root since it's used by the project resolver.
Now I'm wrestling with what to call the TOML package—reusetoml feels clunky, but annotation, config, or global could work better depending on Go naming conventions.
Actually, I could just call the package toml within the module, so the import path becomes github.com/git-pkgs/reuse/toml. The tricky part is that the BurntSushi TOML library is also called toml, so I'd need to alias it internally to avoid the naming conflict.
But thinking about this more, maybe a descriptive name makes sense. Since REUSE.toml is really about annotations and metadata, I could call it annotation or keep it simple with reusetoml. Let me reconsider the overall package structure—I'm thinking about organizing it with separate files for different concerns like types, project operations, file coverage checks, and then subpackages for extraction logic and DEP5 handling.
The tricky part is that subpackages like extract and dep5 need access to core types like ReuseInfo, but the root package also imports those subpackages. I'm running into the classic Go circular dependency problem. I could solve this by moving shared types into their own package, having subpackages return raw data that the root converts, or keeping types in root while subpackages avoid importing it.
Looking at the actual usage patterns, glob is only used by toml, and everything else flows through project which depends on extract, dep5, toml, and covered. The cleanest approach would be extracting the types into a separate package so subpackages can import just the types without creating a cycle. I'm seeing a circular dependency problem: the subpackages need to import the root package for shared types, but the root package also needs to import the subpackages to orchestrate them. The cleanest fix is to have the subpackages define their own types and let the root package handle any conversions, or pass types as parameters rather than having subpackages return them directly. Organizing the package structure with internal types...
The issue is that internal packages are only accessible to their parent, so reuse/extract can import reuse/internal without cycles, but external consumers would receive types from that hidden package. I need to think through whether the types should be re-exported from the root reuse package to make them public-facing.
Looking at how the sibling modules handle this, they use type aliases at the root level to re-export from their internal core packages. So I'd follow the same pattern here: define the core types in reuse/internal/core, then create aliases in reuse.go that expose them as reuse.ReuseInfo, reuse.SourceType, etc. This way subpackages reference the internal types directly while consumers use the public aliases.
Now I'm reconsidering the structure—maybe covered.go should also move into its own subpackage rather than staying at the root, keeping the root focused on just the core types and the main Project interface.
But wait, that creates a cycle again since root would need to import the subpackages for their parsers. The internal/core approach is actually the cleanest solution here—it breaks the dependency cycle by having all the public subpackages import from a shared internal types package, while the root can orchestrate everything without circular imports. I'm laying out the full package structure with all the subpackages and their responsibilities—covered handles file coverage checks, internal/core holds the type definitions, extract manages the reuse info extraction logic, dep5 parses DEP5 files, and toml handles TOML parsing with glob matching support.
The key thing is that project.go stays in the root since it orchestrates everything without creating cycles—the root imports subpackages, subpackages import internal/core, and internal/core has no dependencies on anything else. I also need to handle the naming conflict where the toml package imports the external github.com/BurntSushi/toml library by aliasing the import. I'm leaning toward the toml package name with an internal alias to avoid the naming conflict—it's cleaner for users of the library. Now let me draft out the implementation plan.
Good, the sibling modules (manifests, registries) use internal/core for shared types with type aliases in root. That pattern avoids import cycles when subpackages need shared types.
Let me read the current files to map out the dependency graph precisely.
Now let me write the plan.
>ls /Users/andrew/code/git-pkgs/>ls /Users/andrew/code/git-pkgs/reuse/ 2>/dev/null || echo "directory exists but may be empty"The user wants me to implement a Go REUSE parsing library based on a detailed plan. Let me start by understanding the full scope and then work through the build order systematically.
Let me first check if the directory exists and look at any existing files, then start implementing.
I'll need to:
Let me start by exploring what exists and looking at sibling modules for patterns.
Excellent, I now have a thorough understanding of the REUSE spec, the Python reference implementation patterns, and the sibling module conventions. Let me start implementing.
Let me also look at the zizmor workflow from sibling modules.
Now let me check the zizmor workflow pattern and set up the project.
type ReuseInfo struct { LicenseExpressions []string CopyrightNotices []string Contributors []string SourcePath string // where this info came from SourceType SourceType // file-header, dot-license, reuse-toml, dep5}
type SourceType int // FileHeader, DotLicense, ReuseToml, Dep5
type PrecedenceType int // Closest, Aggregate, Overridetype ReuseTOML struct { Version int Annotations []Annotation Source string}
type Annotation struct { Paths []string Precedence PrecedenceType Copyrights []string Licenses []string}type Dep5 struct { Header Dep5Header Files []Dep5FilesParagraph}
type Dep5FilesParagraph struct { Patterns []string Copyright string License string}type Project struct { Root string ReuseTOML *ReuseTOML // nil if not present Dep5 *Dep5 // nil if not present LicenseFiles []string // paths in LICENSES/}