TCP/IP - Solving The C10K With The Thread Per Client Approach

Just browsing Stack Overflow? Help us improve your experience. Sign up for research

1. 1. Home
   2. Questions
   3. Tags
   4. Users
   5. Companies
   6. Labs
   7. Jobs
   8. Discussions
   9. Collectives

   10. Communities for your favorite technologies. Explore all Collectives

2. Teams

   Ask questions, find answers and collaborate at work with Stack Overflow for Teams.

   Try Teams for free Explore Teams
3. Teams

4. Ask questions, find answers and collaborate at work with Stack Overflow for Teams. Explore Teams

Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

Get early access and see previews of new features.

Learn more about Labs TCP/IP - Solving the C10K with the thread per client approach Ask Question Asked 11 years, 4 months ago Modified 9 years, 10 months ago Viewed 21k times 43

After reading the famous C10k article and searching on the web about how things have evolved since it was written, I would like to know if it would be possible for a today's standard server to handle >10000 concurrent connections using a thread per connection (possibly with the help of a pool of threads to avoid the creation/killing process).

Some details that may affect the approach to the problem:

1. Input, intermediate processing and output.
2. Length of each connection.
3. Technical specifications of the server (cores, processors, RAM, etc...)
4. Combining this system with alternative techniques like AIO, polling, green threads, etc...

Obviously I'm not an expert in the matter, so any remarks or advices will be highly appreciated :)

• multithreading
• concurrency
• tcp
• c10k

Share Improve this question Follow asked Jul 11, 2013 at 12:42 Str1101Str1101 89922 gold badges1212 silver badges2222 bronze badges 1

• Very briefly: while it is possible, it is rarely a good idea (with thread switching cost reaching 100K CPU cycles easily, having too many threads gets expensive for no apparent reason). – No-Bugs Hare Commented Jul 26, 2018 at 9:31

Add a comment |

3 Answers 3

Sorted by: Reset to default Highest score (default) Trending (recent votes count more) Date modified (newest first) Date created (oldest first) 74

Absolutely. A standard server can handle more than 10K concurrent connections using the model with one thread per connection. I have build such an application, and five years ago, it was running with more than 50K concurrent connections per process on a standard Linux server. Nowadays, it should be possible to run the same application with more than 250K concurrent connections on current hardware.

There are only a few things to keep in mind:

• Reuse threads by using a thread pool. There is no need to kill threads if they are not used, because the resource usage should be optimized for peak loads.
• Stack size: By default each Linux thread reserves 8 MB for its stack. That sums up to 80 GB for 10K threads. You should set the default stack size to some value between 64k and 512k, which isn't a problem, because most applications don't require deeper call stacks.
• If the connections are short-lived, optimize for new connections by creating several sockets on the same endpoint with the option SO_REUSEPORT.
• Increase the user limits: open files (default 1.024), max user processes
• Increase system limits, e.g. /proc/sys/kernel/pid_max (default 32K), /proc/sys/kernel/threads-max, and /proc/sys/vm/max_map_count (default 65K).

The application mentioned above was initially designed to handle only 2K concurrent connections. However, with the growth in use, we didn't have to make significant changes to the code in order to scale up to 50K connections.

Share Improve this answer Follow edited Jan 18, 2015 at 15:24 answered Jul 21, 2013 at 10:11 nosidnosid 49.9k1313 gold badges115115 silver badges144144 bronze badges 3

• 4 This is essentially using the thread scheduling system as the packet scheduling system: thread scheduler determines which read() to call next depending on which data arrives. It works, obviously, but there are more scalable designs. – Maxim Egorushkin Commented Oct 27, 2016 at 9:42
• @MaximEgorushkin could you mention examples of more scalable design alternatives for reference? How they differ in terms of thread or packet scheduling? What do they trade-off to be more scalable and in which particular conditions? – Reinaldo Junior Commented Aug 27 at 10:12
• @ReinaldoJunior stackoverflow.com/a/17595620/412080 – Maxim Egorushkin Commented Aug 27 at 19:08

Add a comment | 26

You might like a recent follow-up on the subject: The Secret To 10 Million Concurrent Connections -The Kernel Is The Problem, Not The Solution.

Share Improve this answer Follow answered Jul 11, 2013 at 14:08 Maxim EgorushkinMaxim Egorushkin 136k1717 gold badges193193 silver badges286286 bronze badges 3

• 2 What an amazing article man!! I think that I can learn a lot from it, thank you :) – Str1101 Commented Jul 11, 2013 at 20:50
• @Str1101 I found it quite instructive too. – Maxim Egorushkin Commented Jul 12, 2013 at 9:20
• 1 Makes me think of exokernels again. – Armen Michaeli Commented Jun 6, 2017 at 13:43

Add a comment | 5

The usual approaches for servers are either: (a) thread per connection (often with a thread pool), or (b) single threaded with asynchronous IO (often with epoll or kqueue). My thinking is that some elements of these approaches can, and often should, be combined to use asynchronous IO (with epoll or kqueue) and then hand off the connection request to a thread pool to process. This approach would combine the efficient dispatch of asynchronous IO with the parallelism provided by the thread pool.

I have written such a server for fun (in C++) that uses epoll on Linux and kqueue on FreeBSD and OSX along with a thread pool. I just need to run it through its paces for heavy testing, do some code cleanup, and then toss it out on github (hopefully soon).

Share Improve this answer Follow answered Jul 11, 2013 at 14:01 Paul DardeauPaul Dardeau 2,60911 gold badge1414 silver badges1010 bronze badges 4

• 2 Maybe is that I am a little confused, but I thought that epoll was not exactly asynchronous. After some research I found several texts saying that AIO would work better in combination with a thread pool, and epoll with a single thread: Comparing the aio and epoll. – Str1101 Commented Jul 11, 2013 at 20:35
• 1 ...I also have read that there are some difficulties regarding the implementation of AIO, so maybe it is still more profitable to use the epoll + thread pool approach. whats-the-difference-between.... My own idea about it is that it would be possible to use directly one thread of the pool to hand off every connection until the number of simultaneous connections comes close to the maximum number of threads that the system can sustain, and then we could use the... – Str1101 Commented Jul 11, 2013 at 20:40
• 1 epoll/kqueue/AIO system to handle the new incoming connections while all the threads are busy. (p.s.: I would really love to see your server soon, especially if it is written in C++ :) – Str1101 Commented Jul 11, 2013 at 20:44
• @Str1101: epoll is asynchronous per-socket, and AIO is asynchronous per-request. I.e. if you can guarantee that each socket is used ONLY in one specific thread (which BTW is a Good Idea(tm) regardless of using epoll for several reasons, cache locality included), you'll be fine with epoll in multi-threaded environment. </beating dead horse> – No-Bugs Hare Commented Jul 26, 2018 at 9:29

Add a comment |

Your Answer

Reminder: Answers generated by artificial intelligence tools are not allowed on Stack Overflow. Learn more

Thanks for contributing an answer to Stack Overflow!

• Please be sure to answer the question. Provide details and share your research!

But avoid …

• Asking for help, clarification, or responding to other answers.
• Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

Draft saved Draft discarded

Sign up or log in

Sign up using Google Sign up using Email and Password Submit

Post as a guest

Name Email

Required, but never shown

Post Your Answer Discard

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged

• multithreading
• concurrency
• tcp
• c10k

or ask your own question.

• The Overflow Blog
• Your docs are your infrastructure
• Featured on Meta
• More network sites to see advertising test [updated with phase 2]
• We’re (finally!) going to the cloud!
• Call for testers for an early access release of a Stack Overflow extension...

Linked

57 What's the difference between event-driven and asynchronous? Between epoll and AIO? 12 Threads sitting idle = bad? 0 config file for web server to receive GET request in C 1 Boost: multithread performance, reuse of threads/sockets 1 Which is the maximum number of concurrent connections that a socket can handle without considering computational resources? 0 Linux poll with a thread pool and multiple events

Related

4 Multi Threaded TCP Server in C 0 Trying To Implement Concurrent TCP Server and Client in c 1 Using Java threads to add concurrency to a client for a single-threaded TCP socket server 1 Concurrent TCP server using threads 1 Concurrent TCP Server in Tcl 8 multithread server/client implementation in C 1 TCP threaded server/client 0 Having more than one client per thread in c10k 1 TCP/IP Socket-Server Concurrent 0 How to build a TCP connection with multiple clients, c++

Hot Network Questions

• Why is Ukraine's conscription age (still) so high (25)?
• Responsibility for clearing gutters? (Landlord and Tenant Act 1985)
• How much do ebikes actually benefit from ebike specific wheels, tires, and forks?
• What's a good way to append a nonce to ciphertext in Python for AES GCM in Python?
• Deutsche Bahn Berlin: can I use a different departure station?
• Are mental images of mathematical entities persistent?
• Any way to separate objects with the scatter add-on?
• Are prenups legally binding in England?
• Find the Smallest Data Type for a Number
• Why does Japanese not have a native "tu" sound?
• If Occam's razor supports naturalism over theism, then why was William of Ockham, its author, a theist and not a naturalist?
• If the hard problem of consciousness is unanswerable, is it a hard problem or just a bad question?
• Perfect eden - Whence conflict?
• Is Holy Terra Earth?
• Do I need Letter of invitation to Iceland?
• Is ATL-98 Carvair still alive in the US?
• Why is a head called a head?
• How to Draw a Diagram with a Custom Coordinate System and Shaded Areas?
• What happens when a ranger uses Favored Foe with Hunter's Mark?
• What is the proper way to say "voice direction" in German?
• If someone is irresponsible, or weak-willed “their words lack _____?
• Polynomial.java - a Java class for dealing with polynomials with BigDecimal coefficients
• Should I ask for physical recommendation letters now to avoid future issues with professors' availability?
• Does the 90 day window for VWP reset for extended stay in Mexico?

more hot questions Question feed Subscribe to RSS Question feed

To subscribe to this RSS feed, copy and paste this URL into your RSS reader.