Testing Philosophy with File System Interfaces
File system code presents unique testing challenges that many developers struggle with. The traditional approach of writing tests that directly interact with the real file system creates brittle, slow tests that depend on external state. A better approach treats the file system as a dependency that can be abstracted, controlled, and tested in isolation.
Dependency Injection Benefits
The key to testable file system code lies in dependency injection. Instead of hardcoding calls to os.Open() or ioutil.ReadFile(), inject the file system interface as a dependency. This seemingly simple change transforms your code from untestable to highly testable.
// Problematic: Direct file system dependency func ProcessConfigFile(path string) (*Config, error) { data, err := os.ReadFile(path) if err != nil { return nil, err } return parseConfig(data) } // Better: File system as injected dependency func ProcessConfigFile(fsys fs.FS, path string) (*Config, error) { data, err := fs.ReadFile(fsys, path) if err != nil { return nil, err } return parseConfig(data) } This pattern allows you to pass different file system implementations during testing while maintaining the same interface in production code. The production code receives os.DirFS(".") while tests receive mock implementations.
Testable File System Operations
When designing file system operations for testability, focus on three core principles: interface segregation, explicit dependencies, and predictable behavior.
Interface segregation means depending only on the file system capabilities you actually need. If your function only reads files, depend on fs.FS rather than a broader interface that includes write operations. This makes your intentions clear and simplifies testing.
// Only needs read access func LoadTemplate(fsys fs.FS, name string) (*Template, error) { content, err := fs.ReadFile(fsys, name) if err != nil { return nil, fmt.Errorf("loading template %s: %w", name, err) } return parseTemplate(content) } // Needs write access - use different interface func SaveReport(fsys WriteFS, path string, report *Report) error { data, err := report.Marshal() if err != nil { return err } return fsys.WriteFile(path, data, 0644) } Explicit dependencies eliminate hidden file system interactions. Every file system operation should flow through the injected interface, making the code's dependencies obvious to both humans and tests.
Isolation from Real File System
Test isolation prevents tests from interfering with each other and with the host system. When tests manipulate real files, they can leave behind state that affects subsequent test runs or, worse, modify important system files.
Proper isolation means your tests never touch the real file system unless specifically testing integration points. Unit tests work entirely with mock file systems, while integration tests use temporary directories that are cleaned up automatically.
func TestProcessConfigFile(t *testing.T) { // Isolated test using mock file system testFS := fstest.MapFS{ "config.json": &fstest.MapFile{ Data: []byte(`{"setting": "value"}`), }, } config, err := ProcessConfigFile(testFS, "config.json") assert.NoError(t, err) assert.Equal(t, "value", config.Setting) } This approach gives you complete control over the file system state during tests. You define exactly what files exist, their contents, permissions, and modification times. Tests become deterministic and fast because they don't perform actual disk I/O.
The isolation also extends to error conditions. Mock file systems let you simulate permission errors, disk full conditions, and network failures that would be difficult or impossible to reproduce consistently with real file systems.
Testing file system code well requires thinking about the file system as just another dependency to be managed, not as an unchangeable part of the environment. This mindset shift enables the testing patterns and techniques that follow in the subsequent sections.
Using testing/fstest.TestFS
Go's testing/fstest package provides powerful tools for validating file system implementations and ensuring your code works correctly with any fs.FS implementation. The TestFS function acts as a comprehensive test suite that verifies file system behavior against the expected interface contracts.
Validating Custom FS Implementations
When you create custom file system implementations, fstest.TestFS becomes your verification tool. It runs a battery of tests that check edge cases, error conditions, and specification compliance that you might miss in manual testing.
func TestMyCustomFS(t *testing.T) { // Create your custom file system customFS := &MyCustomFS{ files: map[string][]byte{ "file1.txt": []byte("content1"), "dir/file2.txt": []byte("content2"), "empty.txt": []byte(""), }, } // Validate it conforms to fs.FS interface expectations if err := fstest.TestFS(customFS, "file1.txt", "dir/file2.txt", "empty.txt"); err != nil { t.Fatal(err) } } The function tests various scenarios: opening files that exist and don't exist, reading directories, handling path separators correctly, and ensuring error messages follow Go's conventions. It catches subtle bugs that surface only in specific conditions.
For read-write file systems, create additional validation that goes beyond the basic fs.FS interface:
func TestWritableFS(t *testing.T) { fsys := NewMemoryFS() // First validate basic fs.FS compliance err := fstest.TestFS(fsys, "test1.txt", "dir/test2.txt") require.NoError(t, err) // Then test write operations testWriteOperations(t, fsys) testConcurrentAccess(t, fsys) testErrorConditions(t, fsys) } Test Coverage for Edge Cases
fstest.TestFS excels at uncovering edge cases that developers typically overlook. It tests paths with different separators, attempts to open directories as files, checks behavior with empty files, and validates that file info structures contain correct data.
The tool systematically tests path handling, which is notoriously error-prone:
// These are all tested automatically by fstest.TestFS testCases := []string{ "normal-file.txt", "path/to/nested/file.txt", "file-with-dots..txt", "file.with.multiple.dots", "UPPERCASE.TXT", "file with spaces.txt", } It also validates that your file system handles the distinction between files and directories correctly, ensures that fs.FileInfo returns accurate information, and checks that error types match Go's standard patterns.
The edge case testing extends to boundary conditions. What happens when you try to read beyond a file's end? How does your implementation handle zero-length files? Does it correctly report file modification times? TestFS checks all of these scenarios.
Performance Testing Patterns
While fstest.TestFS focuses on correctness, performance testing requires additional patterns. Create benchmarks that measure your file system implementation under realistic workloads:
func BenchmarkFileSystemOperations(b *testing.B) { fsys := createLargeTestFS(1000) // 1000 files filenames := getRandomFilenames(fsys, 100) b.ResetTimer() b.RunParallel(func(pb *testing.PB) { for pb.Next() { name := filenames[rand.Intn(len(filenames))] if file, err := fsys.Open(name); err == nil { io.Copy(io.Discard, file) file.Close() } } }) } Performance testing should cover different access patterns: sequential reads, random access, directory traversal, and concurrent operations. Each pattern stresses different aspects of your implementation.
Memory usage patterns matter as much as speed. Profile your file system implementation to ensure it doesn't leak memory or consume excessive resources:
func TestMemoryUsage(t *testing.T) { fsys := NewMemoryFS() // Load test data for i := 0; i < 1000; i++ { filename := fmt.Sprintf("file%d.txt", i) fsys.WriteFile(filename, make([]byte, 10240), 0644) } // Measure baseline memory var m1 runtime.MemStats runtime.GC() runtime.ReadMemStats(&m1) // Perform operations performRandomAccess(fsys, 10000) // Check for memory leaks var m2 runtime.MemStats runtime.GC() runtime.ReadMemStats(&m2) memGrowth := m2.Alloc - m1.Alloc if memGrowth > 1024*1024 { // 1MB threshold t.Errorf("Excessive memory growth: %d bytes", memGrowth) } } The combination of fstest.TestFS for correctness validation and custom benchmarks for performance creates a comprehensive testing strategy. Your file system implementation gains confidence through systematic verification while maintaining performance characteristics under realistic workloads.
Mock File System Implementations
Mock file systems give you complete control over the testing environment, allowing you to simulate any condition your code might encounter in production. Unlike stubs that return fixed responses, mocks actively track interactions and can change behavior based on the sequence of operations.
In-Memory Test File Systems
In-memory file systems provide the fastest and most controllable testing environment. They eliminate disk I/O completely while maintaining the same interface as real file systems. The fstest.MapFS type serves as an excellent starting point:
func TestFileProcessor(t *testing.T) { testFS := fstest.MapFS{ "input/data.csv": &fstest.MapFile{ Data: []byte("name,age\nAlice,30\nBob,25"), Mode: 0644, ModTime: time.Date(2023, 1, 1, 12, 0, 0, 0, time.UTC), }, "config/settings.json": &fstest.MapFile{ Data: []byte(`{"delimiter": ",", "headers": true}`), }, } processor := NewFileProcessor(testFS) result, err := processor.ProcessData("input/data.csv", "config/settings.json") assert.NoError(t, err) assert.Len(t, result.Records, 2) assert.Equal(t, "Alice", result.Records[0]["name"]) } For more complex scenarios, create custom in-memory implementations that support write operations and state tracking:
type MemoryFS struct { files map[string]*MemoryFile dirs map[string]bool mutex sync.RWMutex calls []string // Track method calls for verification } func (m *MemoryFS) Open(name string) (fs.File, error) { m.mutex.Lock() m.calls = append(m.calls, fmt.Sprintf("Open(%s)", name)) m.mutex.Unlock() m.mutex.RLock() defer m.mutex.RUnlock() if file, exists := m.files[name]; exists { return &MemoryFile{data: file.data, info: file.info}, nil } return nil, &fs.PathError{Op: "open", Path: name, Err: fs.ErrNotExist} } func (m *MemoryFS) WriteFile(name string, data []byte, perm fs.FileMode) error { m.mutex.Lock() defer m.mutex.Unlock() m.calls = append(m.calls, fmt.Sprintf("WriteFile(%s, %d bytes)", name, len(data))) m.files[name] = &MemoryFile{ data: append([]byte(nil), data...), // Copy to prevent external modification info: &MemoryFileInfo{name: name, size: int64(len(data)), mode: perm}, } return nil } This approach lets you verify not just the final state but also the sequence of operations your code performs.
Controlled Error Injection
Mock file systems excel at simulating error conditions that are difficult to reproduce with real file systems. You can trigger specific errors at precise moments to test error handling paths:
type ErrorInjectingFS struct { base fs.FS failAfter int callCount int errorType error } func (e *ErrorInjectingFS) Open(name string) (fs.File, error) { e.callCount++ if e.callCount > e.failAfter { return nil, e.errorType } return e.base.Open(name) } func TestErrorHandling(t *testing.T) { baseFS := fstest.MapFS{ "file1.txt": &fstest.MapFile{Data: []byte("content1")}, "file2.txt": &fstest.MapFile{Data: []byte("content2")}, "file3.txt": &fstest.MapFile{Data: []byte("content3")}, } // Simulate disk full error after 2 successful operations errorFS := &ErrorInjectingFS{ base: baseFS, failAfter: 2, errorType: errors.New("no space left on device"), } processor := NewBatchProcessor(errorFS) files := []string{"file1.txt", "file2.txt", "file3.txt"} results, err := processor.ProcessFiles(files) // Should process first 2 files successfully, then fail assert.Error(t, err) assert.Len(t, results, 2) assert.Contains(t, err.Error(), "no space left on device") } Error injection patterns help you test partial failure scenarios, retry logic, and cleanup operations. You can simulate network timeouts, permission errors, corrupted files, and resource exhaustion.
State Verification Techniques
Mock file systems enable detailed verification of how your code interacts with the file system. Track not just what files were accessed, but when, in what order, and with what parameters:
type TrackedFS struct { base fs.FS operations []Operation mutex sync.Mutex } type Operation struct { Type string Path string Timestamp time.Time Thread int } func (t *TrackedFS) Open(name string) (fs.File, error) { t.mutex.Lock() t.operations = append(t.operations, Operation{ Type: "Open", Path: name, Timestamp: time.Now(), Thread: getCurrentThreadID(), }) t.mutex.Unlock() return t.base.Open(name) } func TestFileAccessPattern(t *testing.T) { tracked := &TrackedFS{base: createTestFS()} processor := NewConcurrentProcessor(tracked) processor.ProcessDirectory("test-dir") // Verify operations occurred in expected order ops := tracked.GetOperations() assert.Equal(t, "Open", ops[0].Type) assert.Equal(t, "test-dir", ops[0].Path) // Verify no concurrent access to same file fileAccess := groupOperationsByFile(ops) for file, accesses := range fileAccess { ensureNoOverlap(t, file, accesses) } } State verification extends beyond simple call tracking. You can verify that files are closed properly, that locks are acquired and released correctly, and that temporary files are cleaned up. Mock file systems give you visibility into aspects of your code's behavior that would be invisible with real file systems.
The power of mock file systems lies in their predictability and observability. Every aspect of the file system's behavior is under your control, and every interaction is visible to your tests. This combination enables thorough testing of complex file system operations that would be brittle or impossible to test otherwise.
Test Data Management
Effective test data management determines whether your file system tests remain maintainable as your codebase grows. Poor test data practices lead to flaky tests, unclear failure messages, and maintenance nightmares. Well-organized test data makes tests self-documenting and reliable.
Embedding Test Fixtures
Go's embed directive provides an elegant solution for including test files directly in your binary. This approach ensures test data travels with your code and eliminates path-related issues across different environments:
//go:embed testdata/* var testDataFS embed.FS func TestConfigParser(t *testing.T) { tests := []struct { name string filename string expected Config wantErr bool }{ { name: "valid config", filename: "testdata/valid-config.json", expected: Config{Host: "localhost", Port: 8080}, }, { name: "invalid json", filename: "testdata/invalid.json", wantErr: true, }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { config, err := ParseConfig(testDataFS, tt.filename) if tt.wantErr { assert.Error(t, err) return } assert.NoError(t, err) assert.Equal(t, tt.expected, *config) }) } } Organize embedded test data in a clear directory structure that mirrors your test cases:
testdata/ ├── configs/ │ ├── valid-config.json │ ├── invalid-syntax.json │ └── missing-required-field.json ├── templates/ │ ├── simple.tmpl │ └── complex-with-partials.tmpl └── datasets/ ├── small-sample.csv └── large-sample.csv This structure makes it obvious which test data belongs to which test scenario. File names should be descriptive enough that you understand their purpose without opening them.
For binary test data or files that change frequently, consider using a hybrid approach:
func loadTestData(t *testing.T, filename string) []byte { // Try embedded data first if data, err := testDataFS.ReadFile("testdata/" + filename); err == nil { return data } // Fall back to external file for development data, err := os.ReadFile(filepath.Join("testdata", filename)) require.NoError(t, err, "failed to load test data: %s", filename) return data } Dynamic Test File Generation
Some tests require generated data rather than static fixtures. Dynamic generation is particularly useful for testing edge cases, large datasets, or scenarios with specific characteristics:
func generateTestCSV(rows int, corruptLine int) []byte { var buf bytes.Buffer buf.WriteString("id,name,email,age\n") for i := 1; i <= rows; i++ { if i == corruptLine { // Intentionally corrupt this line for error testing buf.WriteString(fmt.Sprintf("%d,\"Unclosed Quote,user%d@example.com,25\n", i, i)) } else { buf.WriteString(fmt.Sprintf("%d,User%d,user%d@example.com,%d\n", i, i, i, 20+i%50)) } } return buf.Bytes() } func TestCSVParserWithLargeFile(t *testing.T) { testFS := fstest.MapFS{ "large.csv": &fstest.MapFile{ Data: generateTestCSV(10000, 5000), // Corrupt line 5000 }, } parser := NewCSVParser(testFS) records, err := parser.ParseFile("large.csv") assert.Error(t, err) assert.Contains(t, err.Error(), "line 5000") assert.Len(t, records, 4999) // Should have parsed up to the error } Dynamic generation shines when testing boundary conditions. Generate files that are exactly at size limits, create directory structures with maximum depth, or produce data with specific statistical distributions:
func createDeepDirectoryStructure(depth int) fstest.MapFS { fs := make(fstest.MapFS) var path strings.Builder for i := 0; i < depth; i++ { if i > 0 { path.WriteString("/") } path.WriteString(fmt.Sprintf("dir%d", i)) } // Add a file at the deepest level filePath := path.String() + "/deep-file.txt" fs[filePath] = &fstest.MapFile{ Data: []byte(fmt.Sprintf("File at depth %d", depth)), } return fs } func TestDeepDirectoryHandling(t *testing.T) { maxDepth := 100 testFS := createDeepDirectoryStructure(maxDepth) walker := NewDirectoryWalker(testFS) files, err := walker.FindAllFiles(".") assert.NoError(t, err) assert.Len(t, files, 1) assert.Contains(t, files[0], "deep-file.txt") } Cleanup Strategies
Test cleanup becomes critical when tests create temporary files or modify shared state. Implement cleanup strategies that work reliably even when tests fail or panic:
func TestFileProcessor(t *testing.T) { // Create temporary directory tempDir := t.TempDir() // Automatically cleaned up by testing framework // For more control, use custom cleanup customTempDir, err := os.MkdirTemp("", "test-processor-*") require.NoError(t, err) t.Cleanup(func() { os.RemoveAll(customTempDir) }) // Test operations... processor := NewFileProcessor(os.DirFS(tempDir)) result, err := processor.ProcessFiles([]string{"input.txt"}) assert.NoError(t, err) assert.NotNil(t, result) } For tests that modify global state or shared resources, use cleanup functions that restore the original state:
func TestWithModifiedEnvironment(t *testing.T) { // Save original environment originalHome := os.Getenv("HOME") originalConfig := os.Getenv("CONFIG_PATH") t.Cleanup(func() { os.Setenv("HOME", originalHome) os.Setenv("CONFIG_PATH", originalConfig) }) // Modify environment for test testHome := t.TempDir() os.Setenv("HOME", testHome) os.Setenv("CONFIG_PATH", filepath.Join(testHome, "config")) // Run test with modified environment config, err := LoadUserConfig() assert.NoError(t, err) assert.Equal(t, testHome, config.HomeDirectory) } Well-managed test data eliminates a major source of test brittleness. When tests fail, you want to debug your application logic, not struggle with missing files or corrupted test data. The patterns above ensure your test data remains an asset rather than a liability as your test suite grows.
Integration Testing Approaches
Integration tests verify that your file system code works correctly with real file systems and external dependencies. While unit tests with mocks validate logic in isolation, integration tests catch issues that emerge from the interaction between your code and the actual environment it will run in.
Temporary Directory Usage
Integration tests need isolated environments that don't interfere with the host system or other tests. Go's testing package provides t.TempDir() which creates temporary directories that are automatically cleaned up:
func TestFileBackupIntegration(t *testing.T) { // Create isolated test environment sourceDir := t.TempDir() backupDir := t.TempDir() // Set up test files testFiles := map[string]string{ "document.txt": "Important document content", "config/app.json": `{"version": "1.0", "debug": true}`, "data/records.csv": "id,name,value\n1,test,100\n2,prod,200", } for path, content := range testFiles { fullPath := filepath.Join(sourceDir, path) err := os.MkdirAll(filepath.Dir(fullPath), 0755) require.NoError(t, err) err = os.WriteFile(fullPath, []byte(content), 0644) require.NoError(t, err) } // Test the backup operation backup := NewFileBackup() err := backup.CreateBackup(sourceDir, backupDir) require.NoError(t, err) // Verify backup contents for path, expectedContent := range testFiles { backupPath := filepath.Join(backupDir, path) actualContent, err := os.ReadFile(backupPath) require.NoError(t, err) assert.Equal(t, expectedContent, string(actualContent)) } // Verify file permissions and metadata sourceInfo, err := os.Stat(filepath.Join(sourceDir, "document.txt")) require.NoError(t, err) backupInfo, err := os.Stat(filepath.Join(backupDir, "document.txt")) require.NoError(t, err) assert.Equal(t, sourceInfo.Mode(), backupInfo.Mode()) assert.Equal(t, sourceInfo.Size(), backupInfo.Size()) } Temporary directories provide true isolation while testing real file system behavior. This catches issues like path separator handling, permission problems, and filesystem-specific limitations that mocks cannot simulate.
Real File System Testing
Some functionality can only be properly tested against real file systems. File locking, atomic operations, and filesystem-specific features require integration testing:
func TestConcurrentFileAccess(t *testing.T) { if testing.Short() { t.Skip("Skipping integration test in short mode") } tempDir := t.TempDir() testFile := filepath.Join(tempDir, "shared.txt") // Create initial file err := os.WriteFile(testFile, []byte("initial content\n"), 0644) require.NoError(t, err) const numWriters = 5 const writesPerWriter = 10 var wg sync.WaitGroup errChan := make(chan error, numWriters) // Launch concurrent writers for i := 0; i < numWriters; i++ { wg.Add(1) go func(writerID int) { defer wg.Done() for j := 0; j < writesPerWriter; j++ { line := fmt.Sprintf("Writer %d, line %d\n", writerID, j) if err := appendToFile(testFile, line); err != nil { errChan <- err return } time.Sleep(time.Millisecond) // Small delay to increase contention } }(i) } wg.Wait() close(errChan) // Check for errors for err := range errChan { t.Errorf("Concurrent write error: %v", err) } // Verify file integrity content, err := os.ReadFile(testFile) require.NoError(t, err) lines := strings.Split(string(content), "\n") expectedLines := 1 + (numWriters * writesPerWriter) // Initial + all writes assert.Equal(t, expectedLines, len(lines)) } func appendToFile(filename, content string) error { file, err := os.OpenFile(filename, os.O_APPEND|os.O_WRONLY, 0644) if err != nil { return err } defer file.Close() _, err = file.WriteString(content) return err } Real file system testing reveals race conditions, locking issues, and platform-specific behaviors that are impossible to catch with mocked implementations.
Cross-Platform Considerations
File system behavior varies significantly across operating systems. Integration tests should account for these differences or explicitly test platform-specific behavior:
func TestPathHandling(t *testing.T) { tempDir := t.TempDir() tests := []struct { name string inputPath string expectError bool skipOnOS string }{ { name: "normal path", inputPath: "normal-file.txt", }, { name: "path with spaces", inputPath: "file with spaces.txt", }, { name: "long filename", inputPath: strings.Repeat("a", 255), // Max filename length on most filesystems }, { name: "invalid characters", inputPath: "file<>|.txt", expectError: true, skipOnOS: "linux", // Linux allows these characters }, { name: "case sensitivity test", inputPath: "CaseSensitive.TXT", }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { if tt.skipOnOS == runtime.GOOS { t.Skipf("Skipping test on %s", runtime.GOOS) } filePath := filepath.Join(tempDir, tt.inputPath) err := os.WriteFile(filePath, []byte("test content"), 0644) if tt.expectError { assert.Error(t, err) } else { assert.NoError(t, err) // Verify file can be read back content, readErr := os.ReadFile(filePath) assert.NoError(t, readErr) assert.Equal(t, "test content", string(content)) } }) } } func TestSymlinkHandling(t *testing.T) { if runtime.GOOS == "windows" { t.Skip("Symlink test requires Unix-like system") } tempDir := t.TempDir() // Create target file targetFile := filepath.Join(tempDir, "target.txt") err := os.WriteFile(targetFile, []byte("target content"), 0644) require.NoError(t, err) // Create symlink linkFile := filepath.Join(tempDir, "link.txt") err = os.Symlink(targetFile, linkFile) require.NoError(t, err) // Test reading through symlink walker := NewDirectoryWalker() files, err := walker.ListFiles(tempDir) require.NoError(t, err) // Should find both target and link assert.Len(t, files, 2) // Verify link behavior linkInfo, err := os.Lstat(linkFile) require.NoError(t, err) assert.True(t, linkInfo.Mode()&os.ModeSymlink != 0) } Platform-specific tests help ensure your code works correctly across different environments. Use build tags or runtime checks to handle platform differences gracefully.
Integration tests bridge the gap between unit tests and production deployment. They catch environment-specific issues while maintaining enough control to be reliable and repeatable. The key is finding the right balance between real-world conditions and test stability.
Advanced Testing Patterns
Advanced testing patterns push beyond traditional unit and integration tests to discover edge cases and performance characteristics that conventional testing might miss. These techniques are particularly valuable for file system code, which often handles unpredictable inputs and operates under varying system conditions.
Property-Based Testing
Property-based testing generates random inputs to verify that your code maintains certain invariants regardless of the specific data it processes. For file system operations, this approach excels at finding edge cases in path handling, data processing, and error recovery:
func TestFilePathNormalization(t *testing.T) { property := func(pathComponents []string) bool { if len(pathComponents) == 0 { return true // Skip empty input } // Filter out empty components and those with null bytes validComponents := make([]string, 0, len(pathComponents)) for _, component := range pathComponents { if component != "" && !strings.Contains(component, "\x00") { validComponents = append(validComponents, component) } } if len(validComponents) == 0 { return true } // Build path and normalize it rawPath := strings.Join(validComponents, "/") normalized := NormalizePath(rawPath) // Properties that should always hold: // 1. Normalized path should not contain double slashes if strings.Contains(normalized, "//") { t.Logf("Double slash found in normalized path: %s", normalized) return false } // 2. Should not end with slash unless it's root if len(normalized) > 1 && strings.HasSuffix(normalized, "/") { t.Logf("Unexpected trailing slash: %s", normalized) return false } // 3. Should be idempotent - normalizing twice gives same result if NormalizePath(normalized) != normalized { t.Logf("Not idempotent: %s != %s", normalized, NormalizePath(normalized)) return false } return true } // Run property test with random inputs for i := 0; i < 1000; i++ { components := generateRandomPathComponents(rand.Intn(10) + 1) if !property(components) { t.Fatalf("Property violation with input: %v", components) } } } func generateRandomPathComponents(count int) []string { components := make([]string, count) chars := "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_." for i := 0; i < count; i++ { length := rand.Intn(20) + 1 component := make([]byte, length) for j := range component { component[j] = chars[rand.Intn(len(chars))] } components[i] = string(component) } return components } Property-based testing for file operations can verify data integrity across various transformations:
func TestFileCompressionRoundTrip(t *testing.T) { compressor := NewFileCompressor() for i := 0; i < 100; i++ { // Generate random data with various characteristics originalData := generateRandomData(rand.Intn(10000) + 1) // Compress and decompress compressed, err := compressor.Compress(originalData) require.NoError(t, err) decompressed, err := compressor.Decompress(compressed) require.NoError(t, err) // Property: round-trip should preserve data exactly assert.Equal(t, originalData, decompressed, "Round-trip failed for data of length %d", len(originalData)) // Property: compressed size should be reasonable ratio := float64(len(compressed)) / float64(len(originalData)) assert.Less(t, ratio, 2.0, "Compression ratio too high: %f for data length %d", ratio, len(originalData)) } } func generateRandomData(size int) []byte { data := make([]byte, size) // Mix different patterns to test various compression scenarios switch rand.Intn(4) { case 0: // Random data rand.Read(data) case 1: // Repeated patterns pattern := []byte("ABCDEFGH") for i := range data { data[i] = pattern[i%len(pattern)] } case 2: // Mostly zeros with some random for i := 0; i < size/10; i++ { data[rand.Intn(size)] = byte(rand.Intn(256)) } case 3: // Text-like data chars := "abcdefghijklmnopqrstuvwxyz \n\n" for i := range data { data[i] = chars[rand.Intn(len(chars))] } } return data } Fuzzing File System Operations
Fuzzing automatically generates test inputs to find crashes, panics, and unexpected behaviors. Go's built-in fuzzing support makes it easy to fuzz file system operations:
func FuzzFilePathParser(f *testing.F) { // Seed corpus with known interesting cases f.Add("/path/to/file.txt") f.Add("../../../etc/passwd") f.Add("file with spaces.txt") f.Add("file\x00with\x00nulls") f.Add(strings.Repeat("a", 1000)) f.Fuzz(func(t *testing.T, input string) { // The function should never panic, regardless of input defer func() { if r := recover(); r != nil { t.Errorf("ParseFilePath panicked with input %q: %v", input, r) } }() result, err := ParseFilePath(input) if err == nil { // If parsing succeeded, result should be valid if result.Dir == "" && result.Base == "" { t.Errorf("ParseFilePath returned empty result for valid input: %q", input) } // Reconstructed path should be equivalent to original reconstructed := filepath.Join(result.Dir, result.Base) if !pathsEquivalent(input, reconstructed) { t.Errorf("Path reconstruction mismatch: %q -> %q", input, reconstructed) } } }) } func FuzzFileReader(f *testing.F) { f.Add([]byte("normal file content")) f.Add([]byte("")) f.Add(make([]byte, 10000)) // Large file f.Add([]byte("\x00\x01\x02\xff")) // Binary data f.Fuzz(func(t *testing.T, data []byte) { // Create temporary file with fuzz data tempFile := filepath.Join(t.TempDir(), "fuzz-test.dat") err := os.WriteFile(tempFile, data, 0644) require.NoError(t, err) // Test file reader with arbitrary data reader := NewFileReader() // Should handle any data without crashing content, err := reader.ReadFile(tempFile) if err != nil { // Error is acceptable, but should not panic return } // If read succeeded, content should match original assert.Equal(t, data, content) }) } Performance Benchmarking
Performance benchmarks measure how your file system code behaves under different load conditions. Effective benchmarks test realistic scenarios and measure relevant metrics:
func BenchmarkFileOperations(b *testing.B) { tempDir := b.TempDir() // Test different file sizes fileSizes := []int{1024, 10240, 102400, 1024000} // 1KB to 1MB for _, size := range fileSizes { b.Run(fmt.Sprintf("size_%d", size), func(b *testing.B) { data := make([]byte, size) rand.Read(data) b.ResetTimer() b.ReportAllocs() for i := 0; i < b.N; i++ { filename := filepath.Join(tempDir, fmt.Sprintf("bench_%d.dat", i)) // Measure write performance err := os.WriteFile(filename, data, 0644) if err != nil { b.Fatal(err) } // Measure read performance _, err = os.ReadFile(filename) if err != nil { b.Fatal(err) } // Clean up os.Remove(filename) } // Report throughput bytesPerOp := int64(size * 2) // Read + write b.SetBytes(bytesPerOp) }) } } func BenchmarkConcurrentFileAccess(b *testing.B) { tempDir := b.TempDir() numWorkers := runtime.NumCPU() b.ResetTimer() b.RunParallel(func(pb *testing.PB) { workerID := atomic.AddInt64(&workerCounter, 1) data := make([]byte, 1024) rand.Read(data) for pb.Next() { filename := filepath.Join(tempDir, fmt.Sprintf("worker_%d_%d.dat", workerID, time.Now().UnixNano())) // Write file err := os.WriteFile(filename, data, 0644) if err != nil { b.Fatal(err) } // Read it back _, err = os.ReadFile(filename) if err != nil { b.Fatal(err) } // Clean up os.Remove(filename) } }) } var workerCounter int64 func BenchmarkDirectoryTraversal(b *testing.B) { // Create directory structure for testing tempDir := b.TempDir() createTestDirectoryStructure(tempDir, 5, 10) // 5 levels deep, 10 files per level walker := NewDirectoryWalker() b.ResetTimer() for i := 0; i < b.N; i++ { files, err := walker.WalkDirectory(tempDir) if err != nil { b.Fatal(err) } // Ensure we found expected number of files if len(files) < 50 { b.Fatalf("Expected at least 50 files, found %d", len(files)) } } } Advanced testing patterns reveal problems that traditional testing approaches miss. Property-based testing finds edge cases in your logic, fuzzing discovers inputs that cause crashes, and performance benchmarks ensure your code scales appropriately. These techniques require more investment than basic unit tests, but they provide confidence that your file system code handles the full range of conditions it will encounter in production.
Top comments (0)