DEV Community

Divya Darshana
Divya Darshana

Posted on

Building a Server-Rendered Dev.to scraper : HTML Templating, API Integration, and Pagination

#go #webdev #productivity #softwaredevelopment

Watch Tutorial Video

Click to watch the companion tutorial video

Why Server-Side Rendering (SSR) Matters

In today's web development landscape dominated by client-side frameworks, server-side rendering offers compelling advantages:

  1. Instant Page Loads: HTML arrives ready-to-display
  2. SEO Superiority: Search engines easily crawl content
  3. Simplified Architecture: Single codebase manages both API and UI
  4. Resource Efficiency: Reduced client-side JavaScript overhead

GoFr, an opinionated Go framework, makes SSR implementation remarkably straightforward. Let's build a Dev.to article reader that demonstrates these principles in action.

Project Setup: Laying the Foundation

1. Initializing the GoFr Application (main.go) 

package main import ( "encoding/json" "gofr.dev/pkg/gofr" "strconv" ) type Article struct { Title string `json:"title"` URL string `json:"url"` Description string `json:"description"` } type PageData struct { Articles []Article Tag string Page int } func main() { // Initialize GoFr application app := gofr.New() // Configure Dev.to API service app.AddHTTPService("dev-articles", "https://dev.to/api") // Register route handler app.GET("/dev-articles", FetchArticles) // Serve static files (CSS, templates) app.AddStaticFiles("/", "./static") // Start the server app.Run() }
Enter fullscreen mode Exit fullscreen mode

Key Components Explained:

  • Service Configuration:AddHTTPService creates a pre-configured HTTP client for the Dev.to API, handling connection pooling and timeouts automatically.

  • Route Handling:The GET method associates the /dev-articles endpoint with our FetchArticles handler.

  • Static Assets:AddStaticFiles serves CSS and templates from the static directory, essential for our SSR approach.

Core Business Logic: The Article Handler

2. Implementing the FetchArticles Handler 

func FetchArticles(ctx *gofr.Context) (any, error) { // Get search parameters with defaults tag := ctx.Param("tag") if tag == "" { tag = "go" // Default to Go articles } page, _ := strconv.Atoi(ctx.Param("page")) if page < 1 { page = 1 // Ensure minimum page number } // Fetch articles from Dev.to API service := ctx.GetHTTPService("dev-articles") resp, err := service.Get(ctx, "/articles", map[string]interface{}{ "tag": tag, "page": page, "per_page": 4, // Optimal for initial load }) if err != nil { return nil, err // Handle API errors } defer resp.Body.Close() // Parse API response var articles []Article if err := json.NewDecoder(resp.Body).Decode(&articles); err != nil { return nil, err // Handle parsing errors } // Render template with data return gofr.Template{ Data: PageData{articles, tag, page}, Name: "devTo.html", }, nil }
Enter fullscreen mode Exit fullscreen mode

Architectural Decisions:

  1. Parameter Handling

    • Default values ensure consistent behavior
    • Type conversion guards against invalid inputs
    • per_page=4 balances content density and performance

  2. Error Handling

    • Automatic error propagation through GoFr's middleware
    • Clean separation of concerns between API and rendering

  3. Service Abstraction

    • HTTP service configuration centralized in main.go
    • Easy to swap API endpoints or add caching later

Presentation Layer: HTML Templating

3. Template Implementation (static/devTo.html) 

<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Go Articles from Dev.to</title> <link rel="stylesheet" href="/style.css"> <script> // Hybrid pagination: Client-side navigation with server-side validation function updatePage(delta) { const params = new URLSearchParams(window.location.search) let page = parseInt(params.get('page') || {{.Page}} let tag = params.get('tag') || '{{.Tag}}' page = Math.max(1, page + delta) window.location.href = `/dev-articles?tag=${tag}&page=${page}` } </script> </head> <body> <div id="content"> <h1>📰 Latest Go Articles on Dev.to</h1> <!-- Search Form --> <form method="GET" action="/dev-articles"> <input type="text" name="tag" placeholder="Tag (e.g. go)" value="{{.Tag}}" /> <button type="submit">Search</button> </form> <!-- Article Listing --> {{range .Articles}} <article class="post"> <h2><a href="{{.URL}}" target="_blank">{{.Title}}</a></h2> <p class="description">{{.Description}}</p> </article> {{end}} <!-- Pagination Controls --> <div class="pagination"> {{if gt .Page 1}} <button onclick="updatePage(-1)">⬅️ Prev</button> {{else}} <button disabled>⬅️ Prev</button> {{end}} <span class="page-number">Page {{.Page}}</span> <button onclick="updatePage(1)">Next ➡️</button> </div> </div> </body> </html>
Enter fullscreen mode Exit fullscreen mode

Template Features:

  1. Dynamic Content Binding

    • {{.Articles}} loop renders server-fetched content
    • {{.Tag}} and {{.Page}} maintain state between requests

  2. Progressive Enhancement

    • Search form works without JavaScript
    • Pagination uses minimal client-side scripting

  3. Security Best Practices

    • Auto-escaping prevents XSS vulnerabilities
    • target="_blank" with rel="noopener" (implicit in Go templates)

Styling: CSS Implementation (static/style.css) 

#content { max-width: 800px; margin: 0 auto; padding: 2rem; font-family: system-ui, sans-serif; } article.post { background: #fff; border-radius: 8px; padding: 1.5rem; margin: 1rem 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1); } .pagination { margin-top: 2rem; display: flex; gap: 1rem; justify-content: center; align-items: center; } button { padding: 0.5rem 1rem; background: #3182ce; color: white; border: none; border-radius: 4px; cursor: pointer; } button:disabled { background: #a0aec0; cursor: not-allowed; } .page-number { font-weight: 500; color: #4a5568; }
Enter fullscreen mode Exit fullscreen mode

Design Philosophy:

  1. Modern Aesthetic

    • System font stack for native feel
    • Subtle shadows and rounded corners

  2. Responsive Layout

    • Max-width constraint for readability
    • Flexible spacing using rem units

  3. Accessibility Focus

    • High contrast colors
    • Clear button states (hover/disabled)

Running the Application

  1. Install Dependencies 
 go mod init your-module-name go get gofr.dev/pkg/gofr
Enter fullscreen mode Exit fullscreen mode
  1. Directory Structure 
├── templates/ │ └── devTo.html # HTML templates ├── static/ │ └── style.css # Static assets └── main.go # Application entrypoint
Enter fullscreen mode Exit fullscreen mode
  1. Start the Server 
 go run main.go
Enter fullscreen mode Exit fullscreen mode
  1. Access in Browser Visit http://localhost:8000/dev-articles?tag=go&page=1

Why GoFr Shines in Production

  1. Built-in Observability

    • Automatic request logging
    • Metrics endpoint at /metrics

  2. Configuration Management

 # configs/config.yml services: dev-articles: url: https://dev.to/api timeout: 3s
Enter fullscreen mode Exit fullscreen mode

  1. Horizontal Scaling

    • Stateless architecture
    • Native Kubernetes support

  2. Error Handling

    • Centralized error recovery
    • Structured logging

Next Steps: From Prototype to Production

  1. Add Caching Layer 
 // Redis integration example cached, err := ctx.Redis.Get(ctx, "articles:"+tag+":"+page)
Enter fullscreen mode Exit fullscreen mode
  1. Implement Rate Limiting 
 app.UseMiddleware(ratelimit.New(100)) // 100 requests/minute
Enter fullscreen mode Exit fullscreen mode
  1. Add Health Checks 
 app.GET("/health", func(ctx *gofr.Context) (any, error) { return "OK", nil })
Enter fullscreen mode Exit fullscreen mode
  1. Error Boundaries 
 {{if .Error}} <div class="error-alert"> ⚠️ Failed to load articles: {{.Error}} </div> {{end}}
Enter fullscreen mode Exit fullscreen mode

Conclusion: The Power of Simplicity

This project demonstrates how Go and GoFr enable building modern web applications with:

✅ Full server-side rendering

✅ Clean architecture

✅ Production-ready features

✅ Minimal dependencies

GitHub Repository:

https://github.com/coolwednesday/gofr-template-rendering-example

Ready to Go Further?

  • Explore GoFr's documentation for database integration
  • Implement user authentication using JWT middleware
  • Add server-side caching for API responses

The simplicity of Go combined with GoFr's powerful abstractions makes this stack ideal for projects ranging from small internal tools to large-scale content platforms.

Happy coding! 🚀

Top comments (0)