 | Hacker Newsnew | past | comments | ask | show | jobs | submit | login |
|
|
| nivertech on March 4, 2011 | parent | context | favorite | on: Do C10K testing with gevent 1. I already solved C1M problem, so C10K and C100K looks like a non-problem to me. EDIT: metabrew has very good tutorial - which should be a starting point for everybody new to this space! 2. The problem is, that Amazon EC2 doesn't support multiple IP aliases per instance. To generate 1M connections, I had to launch a cluster of 17 instances to generate 1M clients. On physical server with a lot of RAM, you can just create 17 IP aliases. 3. Tsung is a great tool - but hard to control and has it's own limits. So if you need to do large scale loadtesting - better write your own tool. |
 | eugenejen on March 4, 2011 [–] I have a stupid question here. When you have a large memory machine, why can't you serve 1M clients on 1 ip without IP aliases? I asked this question because if one uses 17 instances to handle 1M connections, for each instance is around 50k. I am more interested in knowing the possibility and performance for one machine to handle 1M connections. |
| | nivertech on March 4, 2011 | parent | next [–] the real problem is not to generate 1M outgoing connections, but to handle 1M incoming connections on a single node (C1M), while in addition also doing some meaningful processing. It's a solved problem for me. That is why I don't understand why many high profile realtime web startups (like Convore), still using inferior solutions, like Python eventlib or gevent, which at most let them handle hundreds-to-thousands of clients. EDIT: metabrew has very good tutorial - which should be a starting point for everybody new to this space! |
| | eugenejen on March 4, 2011 | root | parent | next [–] Are you referring to http://www.metabrew.com/article/a-million-user-comet-applica... http://www.metabrew.com/article/a-million-user-comet-applica... http://www.metabrew.com/article/a-million-user-comet-applica... By Richard Jones of the Last.fm? Thank you for your replying. The original C10k from Dan Kegel was a guideline for me when I worked on project during 2003 - 2006. I am wondering what is the latest progress in this area. I am glad this problem is solved to you. I will check out niver tech in future when the problem comes up. |
| | wmf on March 4, 2011 | parent | prev | next [–] In the real world your 1M connections will be coming from ~1M clients, so there's no problem. When doing load testing with one client and one server you run into port number limits; this is most realistically solved by using multiple IP addresses on the client (or multiple clients if your cloud is crippled). |
| | stonemetal on March 4, 2011 | parent | prev [–] Port numbers are 2byte values. To track a connection as an IP:port number combination(the way your typical OS does it) you need multiple IPs to give you enough port numbers. |
| | nivertech on March 4, 2011 | root | parent [–] each socket can be uniquely identified by 4-tuple: {LocalIP,LocalPort,ServerIP,ServerPort} LocalIP:LocalPort -> ServerIP,ServerPort So even if number of ports is limited by 64K, total number of incoming sockets per IP address isn't. |
| | nichol4s on March 4, 2011 | root | parent | next [–] Yes, but when you are load testing a specific server, as in this case, you are bound to a specific serverIp and serverPort. And thus, the total amount of sockets you can have open to that server is limited by the amount of IP's your machine has. |
| | stonemetal on March 7, 2011 | root | parent | prev [–] If we are testing a web server on port 80, such that ServerIP,and port are fixed, then the only way you are getting more than 64K connections is through multiple client IP addresses. |
|
|
| Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact Search: |