DEV Community

Cover image for Best Practise with Concurrency And parrallel Programming in C#
Taki
Taki

Posted on

Best Practise with Concurrency And parrallel Programming in C#

#csharp #programming #tutorial #dotnet

Here are five real-world project-based exercises to master concurrency and parallel programming in C#. Each project comes with an overview, objectives, and hands-on tasks.

🚀 Project 1: High-Performance Web Scraper (Async & Parallel)

Overview

Build a multi-threaded web scraper that fetches data from multiple websites in parallel.

Objectives

✅ Use HttpClient with async/await

✅ Implement Parallel.ForEachAsync() for faster data retrieval

✅ Use ConcurrentBag to store results safely

Implementation Steps

1️⃣ Create a list of URLs to scrape.

2️⃣ Fetch data asynchronously using HttpClient.

3️⃣ Use Parallel.ForEachAsync() for concurrent requests.

4️⃣ Store results in ConcurrentBag.

Code Example 

using System; using System.Collections.Concurrent; using System.Net.Http; using System.Threading.Tasks; class Program { static async Task Main() { string[] urls = { "https://jsonplaceholder.typicode.com/posts/1", "https://jsonplaceholder.typicode.com/posts/2", "https://jsonplaceholder.typicode.com/posts/3" }; ConcurrentBag<string> results = new ConcurrentBag<string>(); await Parallel.ForEachAsync(urls, async (url, _) => { using HttpClient client = new HttpClient(); string content = await client.GetStringAsync(url); results.Add(content); Console.WriteLine($"Fetched data from {url}"); }); Console.WriteLine($"Total responses: {results.Count}"); } }
Enter fullscreen mode Exit fullscreen mode

🔹 Enhancement: Store the results in a database or cache (Redis).

🚀 Project 2: Multi-Threaded File Processor

Overview

Process multiple large files concurrently.

Objectives

✅ Read multiple files asynchronously

✅ Use Parallel.ForEach() to process files in parallel

✅ Implement progress tracking

Implementation Steps

1️⃣ Read file names from a directory.

2️⃣ Use Parallel.ForEach() to process files.

3️⃣ Store results in a thread-safe collection.

Code Example 

using System; using System.Collections.Concurrent; using System.IO; using System.Threading.Tasks; class Program { static void ProcessFile(string filePath) { string content = File.ReadAllText(filePath); Console.WriteLine($"Processed: {filePath} (Length: {content.Length})"); } static void Main() { string[] files = Directory.GetFiles("C:\\Logs", "*.log"); Parallel.ForEach(files, ProcessFile); Console.WriteLine("All files processed."); } }
Enter fullscreen mode Exit fullscreen mode

🔹 Enhancement: Save processed data in a database or cloud storage.

🚀 Project 3: Concurrent Order Processing System (Producer-Consumer)

Overview

Simulate a high-performance order processing system.

Objectives

✅ Implement the Producer-Consumer pattern

✅ Use BlockingCollection<T> for safe multi-threaded queueing

✅ Optimize performance with ThreadPool

Implementation Steps

1️⃣ The Producer generates orders and adds them to a queue.

2️⃣ The Consumer processes orders concurrently.

3️⃣ Use BlockingCollection<T> for thread-safe queueing.

Code Example 

using System; using System.Collections.Concurrent; using System.Threading; using System.Threading.Tasks; class Program { static BlockingCollection<string> orderQueue = new BlockingCollection<string>(); static void OrderProducer() { for (int i = 1; i <= 10; i++) { orderQueue.Add($"Order-{i}"); Console.WriteLine($"Added Order-{i}"); Thread.Sleep(100); } orderQueue.CompleteAdding(); } static void OrderConsumer() { foreach (var order in orderQueue.GetConsumingEnumerable()) { Console.WriteLine($"Processing {order}"); Thread.Sleep(200); } } static void Main() { Task.Run(OrderProducer); Task.Run(OrderConsumer).Wait(); } }
Enter fullscreen mode Exit fullscreen mode

🔹 Enhancement: Implement Kafka or RabbitMQ for real-world distributed order processing.

🚀 Project 4: Parallel Image Processing System

Overview

Apply image processing filters (resize, grayscale, blur) in parallel.

Objectives

✅ Load multiple images from a folder

✅ Use Parallel.ForEach() to apply filters

✅ Save processed images

Implementation Steps

1️⃣ Read images from a directory.

2️⃣ Apply image filters using Parallel.ForEach().

3️⃣ Save processed images to a new folder.

Code Example 

using System; using System.Drawing; using System.Drawing.Imaging; using System.IO; using System.Threading.Tasks; class Program { static void ProcessImage(string filePath) { using Bitmap image = new Bitmap(filePath); string outputPath = Path.Combine("ProcessedImages", Path.GetFileName(filePath)); // Convert to grayscale for (int x = 0; x < image.Width; x++) { for (int y = 0; y < image.Height; y++) { Color pixel = image.GetPixel(x, y); int gray = (pixel.R + pixel.G + pixel.B) / 3; image.SetPixel(x, y, Color.FromArgb(gray, gray, gray)); } } image.Save(outputPath, ImageFormat.Jpeg); Console.WriteLine($"Processed: {filePath}"); } static void Main() { string[] files = Directory.GetFiles("C:\\Images", "*.jpg"); Parallel.ForEach(files, ProcessImage); Console.WriteLine("All images processed."); } }
Enter fullscreen mode Exit fullscreen mode

🔹 Enhancement: Use GPU acceleration (e.g., OpenCV) for faster processing.

🚀 Project 5: Scalable Background Job System for ASP.NET Core

Overview

Build a background task processing system using Hangfire or Worker Services.

Objectives

✅ Schedule background jobs

✅ Process jobs asynchronously

✅ Use dependency injection for services

Implementation Steps

1️⃣ Create a Worker Service in ASP.NET Core.

2️⃣ Use Task.Run() for job execution.

3️⃣ Store job results in a database.

Code Example 

using Microsoft.Extensions.Hosting; using System; using System.Threading; using System.Threading.Tasks; public class BackgroundJob : BackgroundService { protected override async Task ExecuteAsync(CancellationToken stoppingToken) { while (!stoppingToken.IsCancellationRequested) { Console.WriteLine($"Processing job at {DateTime.Now}"); await Task.Delay(5000, stoppingToken); } } } class Program { static async Task Main() { await Host.CreateDefaultBuilder() .ConfigureServices((_, services) => services.AddHostedService<BackgroundJob>()) .RunConsoleAsync(); } }
Enter fullscreen mode Exit fullscreen mode

🔹 Enhancement: Integrate Redis or RabbitMQ for distributed job queues.

🔥 Summary Table of Projects

Project Key Concepts Technologies
Web Scraper Async/Await, Parallel Requests HttpClient, ConcurrentBag
File Processor Parallel.ForEach, File I/O Parallel Programming
Order Processing Producer-Consumer, Thread Safety BlockingCollection, ThreadPool
Image Processing Parallel Image Manipulation Bitmap, Parallel.ForEach
Background Jobs Scalable Task Processing ASP.NET Worker Service

Top comments (2)

Collapse
 
rajesh_verma_69c86ea90ea4 profile image
Rajesh Verma •

Great article! Concurrency and parallel programming in C# can be tricky, but you’ve explained the key concepts clearly. I especially liked your breakdown of Task.Run vs Parallel.ForEach—it’s a common point of confusion.
Keep up the good work! 🚀

Collapse
 
tak089 profile image
Taki •

Thanks bro!!