The $100K Code You Forgot You Wrote

The Day I Lost $50K of Code

Last Tuesday at 3:47 PM, a potential client asked me a simple question: "Can you show us something similar to what we need?"

I knew I had built exactly what they wanted. A real-time data processing system with React frontend, WebSocket connections, and MongoDB integration. I remembered the late nights, the perfect solution I'd crafted, the moment everything clicked.

But I couldn't find it.

Forty-five minutes of frantic searching later, I found the project buried in ~/Desktop/untitled_folder_3/test/final/FINAL_REAL/app. No documentation. No README. The git history showed I'd built it during a 48-hour coding sprint six months ago. The client had moved on to another vendor.

That lost opportunity was worth $50,000.

"Your past code is either an asset or archaeology. The only difference is documentation."

That night, I decided to solve this problem once and for all. What I discovered shocked me: I had 147 Git repositories scattered across my Mac. Over 400,000 lines of code. Dozens of fully functional applications, proof-of-concepts, and client deliverables. Most with no documentation whatsoever.

If you've been coding with AI assistance—using tools like GitHub Copilot, ChatGPT, or Claude—you probably have the same problem. We're shipping code faster than ever, but we're also forgetting it faster than ever. This guide shows you exactly how to reclaim that lost value.

Why Modern Development Creates Code Graveyards

The way we build software has fundamentally changed. AI-assisted development has made us 10x more productive, but it's also created a new problem: velocity without memory.

The Perfect Storm

Several factors have converged to create what I call "code graveyards"—repositories full of valuable, working code that might as well not exist because they're undocumented and unfindable:

The Mac-Specific Challenge

macOS presents unique challenges for code discovery. The operating system's elegance comes with opacity:

• Split volumes: Modern macOS separates system and data volumes, making comprehensive searches complex
• Hidden by default: Dot-files and dot-directories (like .git) are invisible in Finder
• Spotlight limitations: Doesn't index git repositories or code comprehensively
• Permission maze: Full Disk Access, Developer Tools access, and other security features can block scanning

Traditional tools weren't built for this reality. They assume you know where your code is, that it's organized, that it has README files. But modern development—especially AI-assisted development—is messier, faster, and more experimental.

Building the Discovery System

I needed a system that assumed nothing about my organization (or lack thereof) and could handle the reality of modern development. The solution came in two parts: discovery and documentation.

Part 1: The Repository Intelligence Scanner

The first script, repo_intel.py, is like a search party for lost code. It walks your entire filesystem (or targeted directories), finds every .git folder, and builds a comprehensive inventory.

What makes this scanner special is its deep code understanding. It doesn't just count files—it actually parses them:

• For Python: Uses the Abstract Syntax Tree (AST) to find every function, class, and import
• For JavaScript/TypeScript: Regex patterns that handle ES6+, JSX, and TypeScript syntax
• For other files: Counts lines and identifies configuration files

The Magic of Metadata

For each repository, the scanner extracts:

This metadata becomes searchable, sortable, and—most importantly—actionable. You can quickly answer questions like:

• "Where's that React app that used WebSockets?"
• "Which projects use MongoDB?"
• "What's my largest Python project?"
• "Which repos haven't been touched in six months?"

The Output: Your Code Inventory

The scanner generates two types of output:

Repository Path,Remote URL,File Count,Total LOC,Functions,Classes,Primary Language
/Users/you/Projects/workout-app,github.com/you/workout,47,3851,127,12,Python
/Users/you/Desktop/client-poc,,23,1247,64,8,JavaScript
/Users/you/experiments/ml-test,,12,892,31,5,Python

# Repository: workout-app

- **Path:** /Users/you/Projects/workout-app  
- **Remote:** https://github.com/you/workout.git  
- **Primary Language:** Python (68%)  
- **Last Modified:** 2024-03-15  

## Summary
- Files analyzed: 47
- Total LOC: 3,851
- Functions: 127
- Classes: 12

## Key Files

### app/models/workout.py (~234 lines)
- **Classes:** Workout, Exercise, Set
- **Functions:** calculate_volume, get_progress, validate_form
- **Imports:** sqlalchemy, datetime, validators

### app/api/routes.py (~156 lines)
- **Functions:** create_workout, get_workouts, update_progress
- **Imports:** flask, json, auth, models

Generating Deep Technical Documentation with AI

Discovery is only half the battle. Once you find a repository, you need to understand it. This is where repo_function_deepdive.py and AI come in.

The Challenge of Code Comprehension

Reading old code is like archaeology. You see the what but not the why. Function names hint at purpose, but the broader architecture, the design decisions, the tradeoffs—all of that context is lost.

Traditional documentation tools generate reference docs: "Function X takes parameters Y and returns Z." But that's not what you need when revisiting old code. You need the story. You need to understand:

• What problem was this solving?
• How do these pieces fit together?
• What are the key workflows?
• Where are the potential issues?
• How would I extend this?

Enter AI-Powered Analysis

The deep-dive script does something remarkable: it reads your code like a senior engineer would during a code review. It:

The Intelligence Layer

What makes this approach powerful is the multi-level analysis:

The 3,000-Word Deep Dive

For each group of related functions, the system generates a comprehensive analysis. Here's what a typical report covers:

Complete Implementation Guide

Now let's build this system. I'll walk you through every step, from setup to deployment.

Prerequisites

Step 1: Environment Setup

# Create a new directory for the project
mkdir ~/code-discovery-system
cd ~/code-discovery-system

# Create a virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install openai

# Set your OpenAI API key
export OPENAI_API_KEY="sk-your-key-here"

# Create the project structure
mkdir -p scripts reports deep_reports

Step 2: The Repository Scanner

Create scripts/repo_intel.py:

#!/usr/bin/env python3
"""
Repository Intelligence Scanner
Discovers and analyzes Git repositories across your filesystem.
"""

import os
import ast
import csv
import json
import argparse
from pathlib import Path
from datetime import datetime
import re

class RepoScanner:
  def __init__(self, root_paths):
      self.root_paths = [Path(p) for p in root_paths]
      self.repos = []
      self.code_patterns = {
          '.py': self.parse_python,
          '.js': self.parse_javascript,
          '.ts': self.parse_javascript,
          '.jsx': self.parse_javascript,
          '.tsx': self.parse_javascript,
      }
  
  def scan(self):
      """Walk filesystem and find all Git repositories."""
      for root in self.root_paths:
          print(f"Scanning {root}...")
          for git_dir in root.rglob('.git'):
              if git_dir.is_dir():
                  repo_path = git_dir.parent
                  self.analyze_repo(repo_path)
      
      self.generate_reports()
  
  def analyze_repo(self, repo_path):
      """Analyze a single repository."""
      print(f"  Analyzing {repo_path}")
      
      repo_data = {
          'path': str(repo_path),
          'remote_url': self.get_remote_url(repo_path),
          'files': [],
          'total_loc': 0,
          'functions': 0,
          'classes': 0,
          'languages': {}
      }
      
      # Walk through all files
      for file_path in repo_path.rglob('*'):
          if file_path.is_file() and not self.should_skip(file_path):
              file_data = self.analyze_file(file_path, repo_path)
              if file_data:
                  repo_data['files'].append(file_data)
                  repo_data['total_loc'] += file_data.get('loc', 0)
                  repo_data['functions'] += file_data.get('functions', 0)
                  repo_data['classes'] += file_data.get('classes', 0)
                  
                  ext = file_path.suffix
                  repo_data['languages'][ext] = repo_data['languages'].get(ext, 0) + 1
      
      self.repos.append(repo_data)
  
  def analyze_file(self, file_path, repo_path):
      """Analyze a single file for functions, classes, and imports."""
      ext = file_path.suffix.lower()
      
      if ext not in self.code_patterns:
          return None
      
      try:
          content = file_path.read_text(encoding='utf-8', errors='ignore')
          relative_path = file_path.relative_to(repo_path)
          
          file_data = {
              'path': str(relative_path),
              'loc': len(content.splitlines()),
              'functions': 0,
              'classes': 0,
              'imports': []
          }
          
          # Parse based on file type
          parser = self.code_patterns[ext]
          parsed_data = parser(content)
          
          file_data.update(parsed_data)
          return file_data
          
      except Exception as e:
          print(f"    Error parsing {file_path}: {e}")
          return None
  
  def parse_python(self, content):
      """Parse Python file using AST."""
      try:
          tree = ast.parse(content)
          functions = [node for node in ast.walk(tree) if isinstance(node, ast.FunctionDef)]
          classes = [node for node in ast.walk(tree) if isinstance(node, ast.ClassDef)]
          imports = [node for node in ast.walk(tree) if isinstance(node, (ast.Import, ast.ImportFrom))]
          
          return {
              'functions': len(functions),
              'classes': len(classes),
              'imports': [self.format_import(imp) for imp in imports[:10]]  # Limit imports
          }
      except:
          return {'functions': 0, 'classes': 0, 'imports': []}
  
  def parse_javascript(self, content):
      """Parse JavaScript/TypeScript using regex patterns."""
      # Function patterns
      function_patterns = [
          r'function\s+\w+\s*\(',
          r'const\s+\w+\s*=\s*\([^)]*\)\s*=>',
          r'let\s+\w+\s*=\s*\([^)]*\)\s*=>',
          r'var\s+\w+\s*=\s*\([^)]*\)\s*=>',
          r'async\s+function\s+\w+\s*\(',
          r'async\s+\([^)]*\)\s*=>'
      ]
      
      # Class patterns
      class_patterns = [
          r'class\s+\w+',
          r'export\s+class\s+\w+'
      ]
      
      # Import patterns
      import_patterns = [
          r'import\s+.*?from\s+[\'"][^\'"]+[\'"]',
          r'require\s*\(\s*[\'"][^\'"]+[\'"]\s*\)'
      ]
      
      functions = sum(len(re.findall(pattern, content)) for pattern in function_patterns)
      classes = sum(len(re.findall(pattern, content)) for pattern in class_patterns)
      imports = re.findall(import_patterns[0], content) + re.findall(import_patterns[1], content)
      
      return {
          'functions': functions,
          'classes': len(classes),
          'imports': imports[:10]  # Limit imports
      }
  
  def format_import(self, node):
      """Format AST import node as string."""
      if isinstance(node, ast.Import):
          return ', '.join(alias.name for alias in node.names)
      elif isinstance(node, ast.ImportFrom):
          module = node.module or ''
          names = ', '.join(alias.name for alias in node.names)
          return f"from {module} import {names}"
      return str(node)
  
  def get_remote_url(self, repo_path):
      """Get Git remote URL if available."""
      try:
          config_path = repo_path / '.git' / 'config'
          if config_path.exists():
              content = config_path.read_text()
              match = re.search(r'url\s*=\s*(.+)', content)
              if match:
                  return match.group(1).strip()
      except:
          pass
      return None
  
  def should_skip(self, file_path):
      """Determine if file should be skipped."""
      skip_patterns = [
          '.git', 'node_modules', '__pycache__', '.DS_Store',
          '.pyc', '.pyo', '.pyd', '.so', '.dylib', '.dll'
      ]
      return any(pattern in str(file_path) for pattern in skip_patterns)
  
  def generate_reports(self):
      """Generate CSV index and markdown reports."""
      # Create reports directory
      reports_dir = Path('reports')
      reports_dir.mkdir(exist_ok=True)
      
      # Generate CSV index
      with open(reports_dir / 'index.csv', 'w', newline='') as f:
          writer = csv.writer(f)
          writer.writerow(['Path', 'Remote URL', 'Files', 'Total LOC', 'Functions', 'Classes', 'Primary Language'])
          
          for repo in self.repos:
              primary_lang = max(repo['languages'].items(), key=lambda x: x[1])[0] if repo['languages'] else 'Unknown'
              writer.writerow([
                  repo['path'],
                  repo['remote_url'] or '',
                  len(repo['files']),
                  repo['total_loc'],
                  repo['functions'],
                  repo['classes'],
                  primary_lang
              ])
      
      # Generate individual markdown reports
      for repo in self.repos:
          self.generate_repo_report(repo, reports_dir)
      
      print(f"\nScan complete! Found {len(self.repos)} repositories.")
      print(f"Reports saved to: {reports_dir.absolute()}")
  
  def generate_repo_report(self, repo, reports_dir):
      """Generate markdown report for a single repository."""
      repo_name = Path(repo['path']).name
      report_path = reports_dir / f"{repo_name}.md"
      
      with open(report_path, 'w') as f:
          f.write(f"# Repository: {repo_name}\n\n")
          f.write(f"- **Path:** {repo['path']}\n")
          if repo['remote_url']:
              f.write(f"- **Remote:** {repo['remote_url']}\n")
          f.write(f"- **Files:** {len(repo['files'])}\n")
          f.write(f"- **Total LOC:** {repo['total_loc']:,}\n")
          f.write(f"- **Functions:** {repo['functions']}\n")
          f.write(f"- **Classes:** {repo['classes']}\n\n")
          
          f.write("## Files\n\n")
          for file_data in sorted(repo['files'], key=lambda x: x['loc'], reverse=True)[:10]:
              f.write(f"### {file_data['path']} (~{file_data['loc']} lines)\n")
              f.write(f"- Functions: {file_data['functions']}\n")
              f.write(f"- Classes: {file_data['classes']}\n")
              if file_data['imports']:
                  f.write(f"- Imports: {', '.join(file_data['imports'][:5])}\n")
              f.write("\n")

def main():
  parser = argparse.ArgumentParser(description='Scan for Git repositories and analyze code')
  parser.add_argument('--roots', nargs='+', default=['~'], 
                     help='Root directories to scan (default: ~)')
  
  args = parser.parse_args()
  
  # Expand user paths
  root_paths = [os.path.expanduser(path) for path in args.roots]
  
  scanner = RepoScanner(root_paths)
  scanner.scan()

if __name__ == '__main__':
  main()

Step 3: The Deep Dive Analyzer

Create scripts/repo_function_deepdive.py:

#!/usr/bin/env python3
"""
Repository Function Deep Dive Analyzer
Generates comprehensive documentation using OpenAI API.
"""

import os
import ast
import json
import argparse
import asyncio
from pathlib import Path
from typing import List, Dict, Any
import openai
from openai import AsyncOpenAI

class DeepDiveAnalyzer:
  def __init__(self, api_key: str, model: str = "gpt-4o-mini"):
      self.client = AsyncOpenAI(api_key=api_key)
      self.model = model
      self.max_workers = 3
      self.group_word_target = 3000
      
  async def analyze_repository(self, repo_path: str, target_files: List[str] = None):
      """Analyze a repository and generate deep-dive reports."""
      repo_path = Path(repo_path)
      
      if not repo_path.exists():
          raise ValueError(f"Repository path does not exist: {repo_path}")
      
      # Find Python and JavaScript files
      code_files = []
      for ext in ['.py', '.js', '.ts', '.jsx', '.tsx']:
          code_files.extend(repo_path.rglob(f'*{ext}'))
      
      if target_files:
          code_files = [f for f in code_files if f.name in target_files]
      
      print(f"Found {len(code_files)} code files to analyze")
      
      # Create output directory
      output_dir = Path('deep_reports') / repo_path.name
      output_dir.mkdir(parents=True, exist_ok=True)
      
      # Analyze each file
      for file_path in code_files:
          await self.analyze_file(file_path, output_dir)
  
  async def analyze_file(self, file_path: Path, output_dir: Path):
      """Analyze a single file and generate deep-dive report."""
      print(f"Analyzing {file_path.name}...")
      
      try:
          content = file_path.read_text(encoding='utf-8', errors='ignore')
          
          if file_path.suffix == '.py':
              functions = self.extract_python_functions(content)
          else:
              functions = self.extract_javascript_functions(content)
          
          if not functions:
              print(f"  No functions found in {file_path.name}")
              return
          
          # Group related functions
          groups = self.group_functions(functions)
          
          # Generate reports for each group
          for i, group in enumerate(groups):
              report = await self.generate_group_report(group, file_path.name)
              
              report_path = output_dir / f"{file_path.stem}_group_{i+1}.md"
              report_path.write_text(report)
              
              print(f"  Generated report: {report_path}")
              
      except Exception as e:
          print(f"  Error analyzing {file_path.name}: {e}")
  
  def extract_python_functions(self, content: str) -> List[Dict[str, Any]]:
      """Extract functions from Python code using AST."""
      try:
          tree = ast.parse(content)
          functions = []
          
          for node in ast.walk(tree):
              if isinstance(node, ast.FunctionDef):
                  func_data = {
                      'name': node.name,
                      'start_line': node.lineno,
                      'end_line': self.get_end_line(node),
                      'code': ast.unparse(node),
                      'calls': self.extract_function_calls(node),
                      'imports': self.extract_imports(node)
                  }
                  functions.append(func_data)
          
          return functions
      except:
          return []
  
  def extract_javascript_functions(self, content: str) -> List[Dict[str, Any]]:
      """Extract functions from JavaScript/TypeScript using regex."""
      import re
      
      functions = []
      lines = content.split('\n')
      
      # Function patterns
      patterns = [
          r'function\s+(\w+)\s*\(',
          r'const\s+(\w+)\s*=\s*\([^)]*\)\s*=>',
          r'let\s+(\w+)\s*=\s*\([^)]*\)\s*=>',
          r'var\s+(\w+)\s*=\s*\([^)]*\)\s*=>',
          r'async\s+function\s+(\w+)\s*\(',
          r'async\s+(\w+)\s*=\s*\([^)]*\)\s*=>'
      ]
      
      for i, line in enumerate(lines):
          for pattern in patterns:
              match = re.search(pattern, line)
              if match:
                  func_name = match.group(1)
                  
                  # Find function end (simplified)
                  start_line = i + 1
                  end_line = self.find_function_end(lines, i)
                  
                  func_data = {
                      'name': func_name,
                      'start_line': start_line,
                      'end_line': end_line,
                      'code': '\n'.join(lines[i:end_line]),
                      'calls': self.extract_js_calls('\n'.join(lines[i:end_line])),
                      'imports': []
                  }
                  functions.append(func_data)
                  break
      
      return functions
  
  def find_function_end(self, lines: List[str], start_idx: int) -> int:
      """Find the end of a function (simplified)."""
      brace_count = 0
      in_function = False
      
      for i in range(start_idx, len(lines)):
          line = lines[i]
          
          if '{' in line:
              brace_count += line.count('{')
              in_function = True
          if '}' in line:
              brace_count -= line.count('}')
          
          if in_function and brace_count == 0:
              return i + 1
      
      return len(lines)
  
  def extract_function_calls(self, node: ast.FunctionDef) -> List[str]:
      """Extract function calls from AST node."""
      calls = []
      for child in ast.walk(node):
          if isinstance(child, ast.Call):
              if isinstance(child.func, ast.Name):
                  calls.append(child.func.id)
      return calls
  
  def extract_js_calls(self, code: str) -> List[str]:
      """Extract function calls from JavaScript code."""
      import re
      calls = re.findall(r'(\w+)\s*\(', code)
      return list(set(calls))
  
  def extract_imports(self, node: ast.FunctionDef) -> List[str]:
      """Extract imports from function scope."""
      imports = []
      for child in ast.walk(node):
          if isinstance(child, (ast.Import, ast.ImportFrom)):
              imports.append(ast.unparse(child))
      return imports
  
  def get_end_line(self, node: ast.AST) -> int:
      """Get the end line of an AST node."""
      for child in ast.walk(node):
          if hasattr(child, 'end_lineno'):
              return child.end_lineno
      return node.lineno + 1
  
  def group_functions(self, functions: List[Dict[str, Any]]) -> List[List[Dict[str, Any]]]:
      """Group related functions based on call relationships."""
      # Simple grouping: functions that call each other
      groups = []
      used = set()
      
      for func in functions:
          if func['name'] in used:
              continue
          
          group = [func]
          used.add(func['name'])
          
          # Find functions that call or are called by this function
          for other_func in functions:
              if other_func['name'] in used:
                  continue
              
              if (func['name'] in other_func['calls'] or 
                  other_func['name'] in func['calls']):
                  group.append(other_func)
                  used.add(other_func['name'])
          
          groups.append(group)
      
      return groups
  
  async def generate_group_report(self, functions: List[Dict[str, Any]], filename: str) -> str:
      """Generate comprehensive report for a group of functions."""
      
      # Prepare the prompt
      prompt = self.build_analysis_prompt(functions, filename)
      
      try:
          response = await self.client.chat.completions.create(
              model=self.model,
              messages=[
                  {"role": "system", "content": "You are a senior software engineer conducting a comprehensive code review. Provide detailed, actionable analysis that would be valuable for understanding, maintaining, and extending the code."},
                  {"role": "user", "content": prompt}
              ],
              max_tokens=4000,
              temperature=0.3
          )
          
          return response.choices[0].message.content
          
      except Exception as e:
          return f"Error generating report: {e}\n\nPrompt: {prompt[:500]}..."
  
  def build_analysis_prompt(self, functions: List[Dict[str, Any]], filename: str) -> str:
      """Build the analysis prompt for OpenAI."""
      
      prompt = f"""Analyze the following group of functions from {filename}. Provide a comprehensive 3,000+ word technical analysis covering:

1. **Executive Summary**: What this code does, its purpose, and business value
2. **Architecture Overview**: How the functions work together, design patterns used
3. **Function Deep Dives**: Detailed analysis of each function's purpose, implementation, and edge cases
4. **Data Flow Analysis**: How data moves through the system
5. **Security Assessment**: Potential vulnerabilities and security considerations
6. **Performance Analysis**: Bottlenecks, optimization opportunities, scaling concerns
7. **Testing Strategy**: Specific test cases and scenarios to cover
8. **Refactoring Opportunities**: Code quality improvements and maintainability enhancements
9. **Future Extensions**: How to extend and scale this functionality

Functions to analyze:

"""
      
      for func in functions:
          prompt += f"""
**Function: {func['name']}**
Lines: {func['start_line']}-{func['end_line']}
Calls: {', '.join(func['calls']) if func['calls'] else 'None'}
Imports: {', '.join(func['imports']) if func['imports'] else 'None'}

```python
{func['code']}
```

"""
      
      prompt += """
Provide a professional, detailed analysis that would be valuable for:
- A developer inheriting this code
- A technical lead reviewing the implementation
- A product manager understanding the system capabilities
- Future maintenance and extension work

Focus on practical insights, actionable recommendations, and deep technical understanding."""
      
      return prompt

async def main():
  parser = argparse.ArgumentParser(description='Generate deep-dive documentation for repositories')
  parser.add_argument('--root', required=True, help='Repository path to analyze')
  parser.add_argument('--files', nargs='*', help='Specific files to analyze')
  parser.add_argument('--model', default='gpt-4o-mini', help='OpenAI model to use')
  parser.add_argument('--max-workers', type=int, default=3, help='Maximum concurrent API calls')
  parser.add_argument('--group-word-target', type=int, default=3000, help='Target words per group report')
  
  args = parser.parse_args()
  
  api_key = os.getenv('OPENAI_API_KEY')
  if not api_key:
      raise ValueError("OPENAI_API_KEY environment variable not set")
  
  analyzer = DeepDiveAnalyzer(api_key, args.model)
  analyzer.max_workers = args.max_workers
  analyzer.group_word_target = args.group_word_target
  
  await analyzer.analyze_repository(args.root, args.files)

if __name__ == '__main__':
  asyncio.run(main())

Step 4: Running the System

Now let's put it all together:

# Step 1: Discover repositories
python3 scripts/repo_intel.py --roots ~/Desktop ~/Documents ~/Projects

# Step 2: Review the index
open reports/index.csv

# Step 3: Pick a repository for deep analysis
python3 scripts/repo_function_deepdive.py \
--root "/Users/you/Projects/workout-app" \
--model gpt-4o-mini \
--max-workers 3

# Step 4: Review the generated documentation
open deep_reports/

What This Means for Your Career

The implications of this discovery extend far beyond just finding forgotten code. This represents a fundamental shift in how we think about our work as developers.

The New Developer Paradigm

We're entering an era where every line of code you write has potential long-term value. The traditional model of "build, ship, forget" is being replaced by "build, ship, document, leverage."

Career Acceleration Opportunities

Proper documentation of your code portfolio opens several career acceleration paths:

The Portfolio Effect

Your documented code portfolio becomes a compounding asset that grows in value over time:

Industry Transformation

This approach has broader implications for the software industry:

The Future of Code Documentation

We're at an inflection point. AI-powered documentation isn't just about catching up with past work—it's about fundamentally changing how we approach code creation and preservation. The developers who embrace this shift will have a significant competitive advantage.

Practical Implementation Details

Let's dive into the actual code that makes this system work. Understanding the implementation will help you customize it for your specific needs.

The Complete Repository Scanner

#!/usr/bin/env python3
"""
repo_intel.py - Repository Intelligence Scanner
Discovers and catalogs all Git repositories on your system.
"""

import os
import ast
import csv
import json
import argparse
import subprocess
from pathlib import Path
from datetime import datetime
import re
from typing import Dict, List, Tuple, Optional

class RepoScanner:
def __init__(self, root_paths: List[str], exclude_patterns: List[str] = None):
    self.root_paths = root_paths
    self.exclude_patterns = exclude_patterns or [
        'node_modules', '.venv', 'venv', '__pycache__', 
        '.cache', 'dist', 'build', '.next', '.git/objects'
    ]
    self.repos_found = []
    
def should_skip_directory(self, path: str) -> bool:
    """Check if directory should be skipped based on exclude patterns."""
    for pattern in self.exclude_patterns:
        if pattern in path:
            return True
    return False
    
def find_git_repos(self) -> List[Dict]:
    """Walk filesystem and find all Git repositories."""
    for root_path in self.root_paths:
        for root, dirs, files in os.walk(root_path):
            # Skip excluded directories
            dirs[:] = [d for d in dirs if not self.should_skip_directory(os.path.join(root, d))]
            
            if '.git' in dirs:
                repo_info = self.analyze_repository(root)
                if repo_info:
                    self.repos_found.append(repo_info)
                    print(f"Found repo: {root}")
                    
    return self.repos_found
    
def analyze_repository(self, repo_path: str) -> Dict:
    """Analyze a single repository."""
    info = {
        'path': repo_path,
        'name': os.path.basename(repo_path),
        'remote': self.get_remote_url(repo_path),
        'branch': self.get_default_branch(repo_path),
        'last_commit': self.get_last_commit_date(repo_path),
        'files': {},
        'total_lines': 0,
        'functions': [],
        'classes': [],
        'imports': set()
    }
    
    # Analyze code files
    for root, dirs, files in os.walk(repo_path):
        dirs[:] = [d for d in dirs if not self.should_skip_directory(os.path.join(root, d))]
        
        for file in files:
            if self.is_code_file(file):
                file_path = os.path.join(root, file)
                file_info = self.analyze_file(file_path)
                if file_info:
                    rel_path = os.path.relpath(file_path, repo_path)
                    info['files'][rel_path] = file_info
                    info['total_lines'] += file_info['lines']
                    info['functions'].extend(file_info['functions'])
                    info['classes'].extend(file_info['classes'])
                    info['imports'].update(file_info['imports'])
                    
    info['imports'] = list(info['imports'])
    return info
    
def is_code_file(self, filename: str) -> bool:
    """Check if file is a code file we should analyze."""
    extensions = {'.py', '.js', '.jsx', '.ts', '.tsx', '.java', '.go', '.rb', '.php'}
    return any(filename.endswith(ext) for ext in extensions)
    
def analyze_file(self, file_path: str) -> Optional[Dict]:
    """Analyze a single code file."""
    try:
        with open(file_path, 'r', encoding='utf-8') as f:
            content = f.read()
            
        lines = len(content.splitlines())
        info = {
            'lines': lines,
            'functions': [],
            'classes': [],
            'imports': []
        }
        
        if file_path.endswith('.py'):
            info.update(self.analyze_python_file(content))
        elif file_path.endswith(('.js', '.jsx', '.ts', '.tsx')):
            info.update(self.analyze_javascript_file(content))
            
        return info
    except Exception as e:
        print(f"Error analyzing {file_path}: {e}")
        return None
        
def analyze_python_file(self, content: str) -> Dict:
    """Extract metadata from Python files using AST."""
    info = {'functions': [], 'classes': [], 'imports': []}
    
    try:
        tree = ast.parse(content)
        
        for node in ast.walk(tree):
            if isinstance(node, ast.FunctionDef):
                info['functions'].append(node.name)
            elif isinstance(node, ast.ClassDef):
                info['classes'].append(node.name)
            elif isinstance(node, ast.Import):
                for alias in node.names:
                    info['imports'].append(alias.name.split('.')[0])
            elif isinstance(node, ast.ImportFrom):
                if node.module:
                    info['imports'].append(node.module.split('.')[0])
                    
    except Exception as e:
        print(f"AST parsing error: {e}")
        
    return info
    
def analyze_javascript_file(self, content: str) -> Dict:
    """Extract metadata from JavaScript/TypeScript files using regex."""
    info = {'functions': [], 'classes': [], 'imports': []}
    
    # Function patterns
    func_patterns = [
        r'function\s+(\w+)\s*\(',
        r'const\s+(\w+)\s*=\s*(?:async\s+)?\(',
        r'const\s+(\w+)\s*=\s*(?:async\s+)?function',
        r'(\w+)\s*:\s*(?:async\s+)?\([^)]*\)\s*(?::\s*\w+)?\s*=>'
    ]
    
    for pattern in func_patterns:
        info['functions'].extend(re.findall(pattern, content))
        
    # Class patterns
    class_patterns = [
        r'class\s+(\w+)',
        r'interface\s+(\w+)',
        r'type\s+(\w+)\s*='
    ]
    
    for pattern in class_patterns:
        info['classes'].extend(re.findall(pattern, content))
        
    # Import patterns
    import_patterns = [
        r'import\s+.*?from\s+[\'"]([^\'"\s]+)[\'"]',
        r'require\s*\([\'"]([^\'"\s]+)[\'"]\)',
        r'import\s*\([\'"]([^\'"\s]+)[\'"]\)'
    ]
    
    for pattern in import_patterns:
        imports = re.findall(pattern, content)
        info['imports'].extend([imp.split('/')[0] for imp in imports])
        
    return info
    
def get_remote_url(self, repo_path: str) -> Optional[str]:
    """Get the remote URL of the repository."""
    try:
        result = subprocess.run(
            ['git', 'remote', 'get-url', 'origin'],
            cwd=repo_path,
            capture_output=True,
            text=True
        )
        return result.stdout.strip() if result.returncode == 0 else None
    except:
        return None
        
def get_default_branch(self, repo_path: str) -> str:
    """Get the default branch name."""
    try:
        result = subprocess.run(
            ['git', 'branch', '--show-current'],
            cwd=repo_path,
            capture_output=True,
            text=True
        )
        return result.stdout.strip() or 'main'
    except:
        return 'main'
        
def get_last_commit_date(self, repo_path: str) -> Optional[str]:
    """Get the date of the last commit."""
    try:
        result = subprocess.run(
            ['git', 'log', '-1', '--format=%ai'],
            cwd=repo_path,
            capture_output=True,
            text=True
        )
        return result.stdout.strip() if result.returncode == 0 else None
    except:
        return None
        
def generate_reports(self, output_dir: str = 'repo_reports'):
    """Generate CSV index and individual Markdown reports."""
    os.makedirs(output_dir, exist_ok=True)
    
    # Generate CSV index
    csv_path = os.path.join(output_dir, 'index.csv')
    with open(csv_path, 'w', newline='') as f:
        writer = csv.DictWriter(f, fieldnames=[
            'name', 'path', 'remote', 'branch', 'last_commit',
            'file_count', 'total_lines', 'functions', 'classes'
        ])
        writer.writeheader()
        
        for repo in self.repos_found:
            writer.writerow({
                'name': repo['name'],
                'path': repo['path'],
                'remote': repo['remote'] or '',
                'branch': repo['branch'],
                'last_commit': repo['last_commit'] or '',
                'file_count': len(repo['files']),
                'total_lines': repo['total_lines'],
                'functions': len(repo['functions']),
                'classes': len(repo['classes'])
            })
            
    # Generate individual Markdown reports
    for repo in self.repos_found:
        self.generate_repo_report(repo, output_dir)
        
    print(f"\nGenerated reports in {output_dir}/")
    print(f"Found {len(self.repos_found)} repositories")
    
def generate_repo_report(self, repo: Dict, output_dir: str):
    """Generate a detailed Markdown report for a single repository."""
    safe_name = repo['name'].replace('/', '_')
    report_path = os.path.join(output_dir, f"{safe_name}.md")
    
    with open(report_path, 'w') as f:
        f.write(f"# Repository: {repo['name']}\n\n")
        f.write(f"- **Path:** {repo['path']}\n")
        f.write(f"- **Remote:** {repo['remote'] or 'None'}\n")
        f.write(f"- **Branch:** {repo['branch']}\n")
        f.write(f"- **Last Commit:** {repo['last_commit'] or 'Unknown'}\n\n")
        
        f.write("## Summary\n")
        f.write(f"- Files analyzed: {len(repo['files'])}\n")
        f.write(f"- Total LOC: {repo['total_lines']:,}\n")
        f.write(f"- Functions: {len(repo['functions'])}\n")
        f.write(f"- Classes: {len(repo['classes'])}\n\n")
        
        if repo['imports']:
            f.write("## Dependencies\n")
            unique_imports = sorted(set(repo['imports']))
            for imp in unique_imports[:20]:  # Top 20 imports
                f.write(f"- {imp}\n")
            if len(unique_imports) > 20:
                f.write(f"- ... and {len(unique_imports) - 20} more\n")
            f.write("\n")
            
        f.write("## Files\n\n")
        for file_path, file_info in sorted(repo['files'].items()):
            f.write(f"### {file_path}\n")
            f.write(f"- Lines: {file_info['lines']}\n")
            if file_info['functions']:
                f.write(f"- Functions: {', '.join(file_info['functions'][:10])}")
                if len(file_info['functions']) > 10:
                    f.write(f" ... and {len(file_info['functions']) - 10} more")
                f.write("\n")
            if file_info['classes']:
                f.write(f"- Classes: {', '.join(file_info['classes'])}\n")
            f.write("\n")

def main():
parser = argparse.ArgumentParser(description='Scan for Git repositories')
parser.add_argument('--root', action='append', help='Root directory to scan')
parser.add_argument('--output', default='repo_reports', help='Output directory')
args = parser.parse_args()

# Default to home directory if no root specified
root_paths = args.root or [str(Path.home())]

scanner = RepoScanner(root_paths)
scanner.find_git_repos()
scanner.generate_reports(args.output)

if __name__ == '__main__':
main()

The Deep-Dive Documentation Generator

#!/usr/bin/env python3
"""
repo_function_deepdive.py - AI-Powered Code Documentation Generator
Generates comprehensive technical documentation using OpenAI.
"""

import os
import ast
import re
import json
import argparse
from pathlib import Path
from typing import Dict, List, Set, Tuple
import openai
from concurrent.futures import ThreadPoolExecutor, as_completed
import time

class FunctionAnalyzer:
def __init__(self, model: str = "gpt-4", max_workers: int = 3):
    self.model = model
    self.max_workers = max_workers
    self.client = openai.OpenAI()
    
def extract_functions_from_file(self, file_path: str) -> List[Dict]:
    """Extract all functions from a code file."""
    with open(file_path, 'r', encoding='utf-8') as f:
        content = f.read()
        
    if file_path.endswith('.py'):
        return self.extract_python_functions(content, file_path)
    elif file_path.endswith(('.js', '.jsx', '.ts', '.tsx')):
        return self.extract_javascript_functions(content, file_path)
    else:
        return []
        
def extract_python_functions(self, content: str, file_path: str) -> List[Dict]:
    """Extract functions from Python code using AST."""
    functions = []
    
    try:
        tree = ast.parse(content)
        lines = content.splitlines()
        
        for node in ast.walk(tree):
            if isinstance(node, ast.FunctionDef):
                # Get function body
                start_line = node.lineno - 1
                end_line = node.end_lineno
                body_lines = lines[start_line:end_line]
                
                # Extract calls made by this function
                calls = []
                for child in ast.walk(node):
                    if isinstance(child, ast.Call):
                        if isinstance(child.func, ast.Name):
                            calls.append(child.func.id)
                        elif isinstance(child.func, ast.Attribute):
                            calls.append(child.func.attr)
                            
                functions.append({
                    'name': node.name,
                    'file': file_path,
                    'start_line': start_line,
                    'end_line': end_line,
                    'body': '\n'.join(body_lines),
                    'calls': list(set(calls)),
                    'decorators': [d.id for d in node.decorator_list if isinstance(d, ast.Name)],
                    'args': [arg.arg for arg in node.args.args]
                })
                
    except Exception as e:
        print(f"Error parsing Python file {file_path}: {e}")
        
    return functions
    
def extract_javascript_functions(self, content: str, file_path: str) -> List[Dict]:
    """Extract functions from JavaScript/TypeScript using regex."""
    functions = []
    lines = content.splitlines()
    
    # Pattern to match various function declarations
    patterns = [
        # Named functions
        (r'^(?:export\s+)?(?:async\s+)?function\s+(\w+)\s*\([^)]*\)', 'function'),
        # Arrow functions assigned to const/let/var
        (r'^(?:export\s+)?(?:const|let|var)\s+(\w+)\s*=\s*(?:async\s+)?\([^)]*\)\s*=>', 'arrow'),
        # Method syntax
        (r'^(?:\s*)(\w+)\s*\([^)]*\)\s*{', 'method'),
        # Class methods
        (r'^(?:\s*)(?:async\s+)?(\w+)\s*\([^)]*\)\s*{', 'class_method')
    ]
    
    for i, line in enumerate(lines):
        for pattern, func_type in patterns:
            match = re.match(pattern, line.strip())
            if match:
                func_name = match.group(1)
                
                # Find the end of the function
                brace_count = 0
                start_line = i
                end_line = i
                
                for j in range(i, len(lines)):
                    brace_count += lines[j].count('{') - lines[j].count('}')
                    if brace_count == 0 and j > i:
                        end_line = j
                        break
                        
                body = '\n'.join(lines[start_line:end_line + 1])
                
                # Extract function calls (simplified)
                call_pattern = r'(\w+)\s*\('
                calls = list(set(re.findall(call_pattern, body)))
                
                functions.append({
                    'name': func_name,
                    'file': file_path,
                    'start_line': start_line,
                    'end_line': end_line,
                    'body': body,
                    'calls': calls,
                    'type': func_type
                })
                
    return functions
    
def build_call_graph(self, functions: List[Dict]) -> Dict[str, Set[str]]:
    """Build a call graph from function list."""
    func_names = {f['name'] for f in functions}
    call_graph = {}
    
    for func in functions:
        callers = set()
        for other_func in functions:
            if func['name'] in other_func['calls'] and func['name'] != other_func['name']:
                callers.add(other_func['name'])
        call_graph[func['name']] = callers
        
    return call_graph
    
def find_connected_components(self, functions: List[Dict], call_graph: Dict[str, Set[str]]) -> List[List[Dict]]:
    """Group functions into connected components based on call relationships."""
    func_map = {f['name']: f for f in functions}
    visited = set()
    components = []
    
    def dfs(func_name: str, component: List[str]):
        if func_name in visited or func_name not in func_map:
            return
        visited.add(func_name)
        component.append(func_name)
        
        # Add functions that call this function
        if func_name in call_graph:
            for caller in call_graph[func_name]:
                dfs(caller, component)
                
        # Add functions that this function calls
        func = func_map[func_name]
        for called in func['calls']:
            if called in func_map:
                dfs(called, component)
                
    for func in functions:
        if func['name'] not in visited:
            component_names = []
            dfs(func['name'], component_names)
            component = [func_map[name] for name in component_names if name in func_map]
            if component:
                components.append(component)
                
    return components
    
def generate_analysis_prompt(self, functions: List[Dict], word_target: int = 3000) -> str:
    """Generate a prompt for OpenAI to analyze the functions."""
    prompt = f"""Analyze this group of related functions and provide a comprehensive technical deep-dive of approximately {word_target} words.

Your analysis should include:

1. **Executive Summary** - High-level overview of what this code does and its primary purpose
2. **Architecture Overview** - How these functions work together, key design patterns used
3. **Detailed Function Analysis** - For each function: purpose, parameters, returns, side effects, complexity
4. **Data Flow** - How data moves through these functions, transformations applied
5. **Dependencies & Integration** - External dependencies, API calls, database interactions
6. **Error Handling** - How errors are handled, potential failure modes
7. **Security Considerations** - Authentication, authorization, input validation, potential vulnerabilities
8. **Performance Analysis** - Bottlenecks, scaling concerns, optimization opportunities
9. **Testing Strategy** - Specific test cases to write, edge cases to consider
10. **Refactoring Opportunities** - Specific improvements to code quality, maintainability
11. **Future Extensions** - How to extend this functionality, integration possibilities

Make the analysis specific and actionable. Include code snippets where helpful. Focus on insights that would help a developer understand and work with this code.

Functions to analyze:

"""
    
    for func in functions:
        prompt += f"\n### Function: {func['name']} (in {os.path.basename(func['file'])})\n"
        prompt += f"```python\n{func['body'][:500]}...\n```\n"  # Truncate long functions
        
    return prompt
    
def analyze_component(self, component: List[Dict], output_dir: str, group_index: int, word_target: int = 3000):
    """Analyze a component and generate documentation."""
    if not component:
        return
        
    # Create output directory
    first_file = os.path.basename(component[0]['file']).replace('.', '_')
    component_dir = os.path.join(output_dir, first_file)
    os.makedirs(component_dir, exist_ok=True)
    
    # Generate analysis
    prompt = self.generate_analysis_prompt(component, word_target)
    
    try:
        response = self.client.chat.completions.create(
            model=self.model,
            messages=[
                {"role": "system", "content": "You are a senior software engineer providing detailed code analysis."},
                {"role": "user", "content": prompt}
            ],
            max_tokens=4000,
            temperature=0.7
        )
        
        analysis = response.choices[0].message.content
        
        # Save analysis
        output_path = os.path.join(component_dir, f"group_{group_index}_analysis.md")
        with open(output_path, 'w') as f:
            f.write(f"# Analysis: {first_file} - Group {group_index}\n\n")
            f.write(f"**Functions analyzed:** {', '.join([f['name'] for f in component])}\n\n")
            f.write("---\n\n")
            f.write(analysis)
            
        print(f"Generated analysis for {first_file} group {group_index}")
        
    except Exception as e:
        print(f"Error generating analysis: {e}")
        
def analyze_repository(self, repo_path: str, output_dir: str = "deep_reports", word_target: int = 3000):
    """Analyze all code files in a repository."""
    all_functions = []
    
    # Extract functions from all code files
    for root, dirs, files in os.walk(repo_path):
        for file in files:
            if self.is_code_file(file):
                file_path = os.path.join(root, file)
                functions = self.extract_functions_from_file(file_path)
                all_functions.extend(functions)
                
    if not all_functions:
        print("No functions found to analyze")
        return
        
    print(f"Found {len(all_functions)} functions to analyze")
    
    # Build call graph and find components
    call_graph = self.build_call_graph(all_functions)
    components = self.find_connected_components(all_functions, call_graph)
    
    print(f"Grouped into {len(components)} connected components")
    
    # Analyze components in parallel
    with ThreadPoolExecutor(max_workers=self.max_workers) as executor:
        futures = []
        for i, component in enumerate(components):
            future = executor.submit(self.analyze_component, component, output_dir, i, word_target)
            futures.append(future)
            
        for future in as_completed(futures):
            future.result()  # This will raise any exceptions that occurred
            
def is_code_file(self, filename: str) -> bool:
    """Check if file is a code file we should analyze."""
    extensions = {'.py', '.js', '.jsx', '.ts', '.tsx'}
    return any(filename.endswith(ext) for ext in extensions)

def main():
parser = argparse.ArgumentParser(description='Generate deep technical documentation for code')
parser.add_argument('--root', required=True, help='Repository root directory')
parser.add_argument('--output', default='deep_reports', help='Output directory')
parser.add_argument('--model', default='gpt-4', help='OpenAI model to use')
parser.add_argument('--max-workers', type=int, default=3, help='Max parallel API calls')
parser.add_argument('--group-word-target', type=int, default=3000, help='Target words per analysis')
args = parser.parse_args()

analyzer = FunctionAnalyzer(model=args.model, max_workers=args.max_workers)
analyzer.analyze_repository(args.root, args.output, args.group_word_target)

if __name__ == '__main__':
main()

Real Results from My Own Machine

When I first ran these scripts on my own machine, I was genuinely shocked by what I found. Here are the actual results:

The Discoveries

Among the forgotten repositories, I found:

The Patterns

Analyzing 147 repositories revealed clear patterns in my coding evolution:

The Value Calculation

Here's how I calculated the $294,000 figure:

Note: This is conservative. It doesn't include the time saved by having reusable code, the learning value, or the compound effect of a documented portfolio.

The Economics of Undocumented Code

Let's talk real numbers. The economics of code documentation go far beyond just the immediate value of the code itself.

The True Cost Model

Every undocumented repository has both direct and opportunity costs:

ROI Calculation

The return on investment for documentation is compelling:

The Compound Effect

Documentation creates compound value over time. Each documented project:

1. Becomes searchable - You can find relevant code in seconds instead of hours
2. Enables reuse - Copy proven solutions instead of reinventing them
3. Demonstrates expertise - Show depth of knowledge to clients and employers
4. Preserves context - Understand why decisions were made, not just what was built
5. Facilitates learning - See your own growth and patterns over time

Market Value Analysis

Based on conversations with recruiters and hiring managers, documented portfolios command premium rates:

The ability to demonstrate not just what you built, but how you think about architecture, security, and scalability, directly translates to higher compensation.

Client Acquisition Impact

For freelancers and consultants, documentation has an even more dramatic effect:

"When I started showing potential clients my documented portfolio with deep technical analysis, my close rate went from 15% to 65%. I also increased my hourly rate by 80% because I could demonstrate expertise, not just claim it."

— Senior Full-Stack Consultant

Security and Privacy Considerations

Before you run these scripts, it's crucial to understand the security and privacy implications.

What Gets Sent Where

Privacy Best Practices

Recommended Exclude Patterns

# Add to your scanner configuration
EXCLUDE_PATTERNS = [
    # Dependencies
    'node_modules',
    'venv',
    '.venv',
    'vendor',
    
    # Build artifacts
    'dist',
    'build',
    '.next',
    'out',
    
    # Sensitive data
    '.env',
    '.env.*',
    'secrets',
    'credentials',
    '*.pem',
    '*.key',
    
    # Customer data
    'customer_data',
    'user_uploads',
    'backups',
    
    # Large files
    '*.sql',
    '*.csv',
    '*.log'
]

Enterprise Considerations

If you're using this in an enterprise environment:

Cost Control

Keep your API costs under control with these strategies:

1. Start small - Analyze one repository at a time
2. Use efficient models - GPT-4 Turbo or GPT-3.5 for initial passes
3. Limit snippet size - Adjust --max-snippet-chars
4. Batch wisely - Group related functions to minimize API calls
5. Monitor usage - Set up billing alerts in your OpenAI account

Your Next Steps

You've read the guide. You understand the value. Now it's time to act.

The 30-Minute Quick Start

The One-Day Investment

If you can dedicate a full day to this, here's the optimal approach:

1. Morning: Discovery
   • Run comprehensive scan of your main development directories
   • Review and categorize all discovered repositories
   • Identify top 10-20 repos worth documenting
2. Afternoon: Documentation
   • Run deep-dive analysis on priority repositories
   • Review generated documentation
   • Create a master index of your work
3. Evening: Action
   • Update your GitHub profile with documented projects
   • Extract reusable components
   • Identify immediate opportunities (client work, products, blog posts)

Making It a Habit

Documentation should become part of your development workflow:

Beyond Documentation

Once you have your code documented, consider these value-amplifying actions:

Conclusion: Your Code Is Your Legacy

Every line of code you write is a reflection of your problem-solving ability, your creativity, and your technical growth. But undocumented, that code might as well not exist.

The tools and techniques in this guide transform your forgotten repositories from digital archaeology into a living, valuable portfolio. They turn "I think I built something like that once" into "Here's exactly how I solved that problem, and here's how we can adapt it for your needs."

In an industry that moves at breakneck speed, where yesterday's framework is today's legacy code, documentation is your competitive advantage. It's proof of your journey, evidence of your capabilities, and a foundation for your future growth.

Your code tells a story. Make sure it's not a mystery novel.