Redis, Memcached, Tokyo Tyrant and MySQL comparision
I wanted to compare the following DBs, NoSQLs and caching solutions for speed and connections. Tested the following
- Redis
- Memcached
- Tokyo Tyrant / Tokyo Cabinet
- MySQL 5.1.40 (MyISAM)
- MySQL 5.1.40 (with Innodb Plugin 1.0.4), compiled into source of MySQL
My test had the following criteria
- 2 client boxes
- All clients connecting to the server using Python
- Used Python’s threads to create concurrency
- Each thread made 10,000 open-close connections to the server
- The server was
- Intel(R) Pentium(R) D CPU 3.00GHz
- Fedora 10 32bit
- Intel(R) Pentium(R) D CPU 3.00GHz
- 2.6.27.38-170.2.113.fc10.i686 #1 SMP
- 1GB RAM
- Used a md5 as key and a value that was saved
- Created an index on the key column of the table
- Each server had SET and GET requests as a different test at same concurrency
Results please !
I wanted to simulate a situation where I had 2 servers (clients) serving my code, which connected to the 1 server (memcached, redis, or whatever). Another thing to note was that I used Python as the client in all the tests, definately the tests would give a different output had I used PHP. Again the test was done to check how well the clients could make and break the connections to the server, and I wanted the overall throughput after making and breaking the connections. I did not monitor the response times. I didnt change absolutely any parameters for the servers, eg didn’t change the innodb_buffer_pool_size or key_buffer_size.
MySQL
MySQL lacked the whole scene terribly, I monitored the MySQL server via the MySQL Administrator and found that hardly there were any conncurrent inserts or selects, I could see the unauthenticated users, which meant that the client had connected to MySQL and was doing a handshake using MySQL authentication (using username and password). As you could see I didn’t even perform the 40 and 60 thread tests.
I truncated the table before I swtiched my tests from MyISAM to InnoDB. And always started the tests from lesser threads. My table was as follows
CREATE TABLE `comp_dump` ( `k` char(32) DEFAULT NULL, `v` char(32) DEFAULT NULL, KEY `ix_k` (`k`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1
NoSQL
For Tokyo Tyrant I used a file.tch as the DB, which is a hash database. I also tried MongoDB as u may find if u have opened the worksheet, But the server kept failing or actually the mongod failed after coming at an unhandled Exception. I found something similar over here. I tried 1.0.1, 1.1.3 and the available Nightly build, but all failed and I lost my patience.
Now what
If you need speed just to fetch a data for a given combination or key, Redis is a solution that you need to look at. MySQL can no way compare to Redis and Memcache. If you find Memcache good enough, you may want to look at Tokyo Tyrant as it does a synchronous writes. But you need to check for your application which server/combination suits you the best. In Marathi there is a saying “मेल्या शिवाय स्वर्ग दिसत नाही”, which means “You can’t see heaven without dieing” or need to do your hard work, can’t escape that 
I’ve attached the source code used to test, if anybody has any doubts, questions feel free to ask
| throughput-get.png | 8.57 KB |
| throughput-set.png | 8.65 KB |
| worksheet.png | 42.36 KB |
| comparision.tar.gz | 7.46 KB |
My previous post Redis, Memcache, Tokyp Tyrant, MySQL comparison. The MySQL was taking a huge time for doing a reverse DNS lookup.
I turned on the skip-name-resolve parameter in the my.cnf and the Throughput of MySQL grew considerably, almost more than double.
Here are the new results.
MyISAM vs InnoDB
Nothing much has changed in the above test. Except for the fact InnoDB starts leading the way when there are high number of concurrent Inserts/Updates or Writes on the table. As seen from the “Set” graph InnoDB starts closing for MyISAM’s write efficiency around 30 concurrent requests and then by 60 concurrent requests its already ahead in throughput of writes – 1284/s against 825/s. Further I had put a watch on processlist and was watching the processess, there were times during MyISAM when the inserts took over 6seconds to finish, which also means that if you are in a need of an application which requires quicker response during heavy loads / heavy concurrency… You need to check the MyISAM vs. InnoDB scenario really closely. At low concurrency MyISAM is well ahead in writes, and in Reads, both MyISAM and InnoDB perform equally well.
Again you need to make sure that you check ur test conditions really well before just taking InnoDB for granted.
| throughput-get2.png | 7.96 KB |
| throughput-set2.png | 8.71 KB |
| worksheet-2.png | 23.31 KB |
| comparision.ods | 29.02 KB |