Monday, April 19, 2010

Cassandra Insert Throughput

** 0.5.1

Test Cluster:
DELL 2950 1*CPU Intel Xeon 5310 (4 cores)
5 nodes
1 node: 2GB heap for Cassandra JVM
4 nodes: 4GB heap for Cassandra JVM

Commit-log and Data stored on same disks.
25 client threads run on 5 nodes.

Data Model:
Keyspace Name = “Test”
Column Family Name = “ABC”
CompareWith for Column = LongType
Column Name = Timestamp (LongType), Value = 400 bytes binary
Billions of keys, thousands of columns.

Partitioner = dht.RandomPartitioner
MemtableSizeInMB = 64MB
ReplicationFactor = 3

Use Thrift Client Interface
Client.insert(..)
Consistency Level (write) = 1

Total inserted 1,076,333,461 columns.
Disk Use: 302GB+283GB+335GB+186GB+276GB=1,382GB (~~400B*1G=400GB *3= 1200GB)

On inserting: 1000 SSTables on each node. The latency of a query is about 1~3 seconds.
Quiet for long time: 10 SSTables (very big files, such as there is one 144GB SSTable data file)
The latency of a query is in ms.

Result: 18,000 columns/second


** 0.6.0
Only 4 nodes.

JVM GC for big heap.
Memory, GC..., always to be the bottleneck and big issue of java-based infrastructure software!
https://issues.apache.org/jira/browse/CASSANDRA-896 (LinkedBlockingQueue issue, fixed in jdk-6u19)

Seems 0.5.1 performed better.
0.6.0 eat more memory.

44 comments:

  1. Big data is now taking the guesswork out of discerning which individuals are the best targets for a particular product. To know more about SAP, Visit Big data training in chennai

    ReplyDelete
  2. I just see the post i am so happy to the communication science post of information's.So I have really enjoyed and reading your blogs for these posts.Any way I’ll be replay for your great thinks and I hope you post again soon...
    Java Training in Chennai

    ReplyDelete
  3. The blog gave me idea about components of selenium. They explained in effective manner. Thanks for sharing it. Keep sharing more blogs.

    Selenium Training in Chennai

    ReplyDelete
  4. This information is impressive; I am inspired with your post writing style & how continuously you describe this topic.


    digital-marketing

    ReplyDelete
  5. I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details.Block Chain Training in chennai

    ReplyDelete
  6. Nice....bigdata is clearly explained..thanks for sharing..keep blogging

    ReplyDelete
  7. I am obliged to you for sharing this piece of information here and updating us with your resourceful guidance. Hope this might benefit many learners. Keep sharing this gainful articles and continue updating us.
    Angularjs Training in Chennai
    Angularjs Training
    Angularjs Training near me
    DevOps Training in Chennai
    DevOps certification Chennai
    DevOps certification

    ReplyDelete
  8. Hello! This is my first visit to your blog! We are a team of volunteers and starting a new initiative in a community in the same niche. Your blog provided us useful information to work on. You have done an outstanding job.

    Advanced AWS Training in Marathahalli |No.1 AWS Training in Marathahalli
    Best AWS Amazon Web Services Training Institute in Chennai | No.1 AWS Training Institutes for Solution Architect in Chennai | Advanced AWS Certification Training in Chennai

    ReplyDelete
  9. I am really enjoying reading your well-written articles. It looks like you spend a lot of effort and time on your blog. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work.
    SEO Training in Chennai
    Digital Marketing Chennai
    Digital Marketing Courses in Chennai
    Digital marketing courses
    SEO Institutes in Chennai
    SEO Course Chennai

    ReplyDelete
  10. Your story is truly inspirational and I have learned a lot from your blog. Much appreciated.

    Python training in bangalore
    Python course in pune
    Python training in bangalore

    ReplyDelete
  11. Wow it is really wonderful and awesome thus it is very much useful for me to understand many concepts and helped me a lot. it is really explainable very well and i got more information from your blog.

    best rpa training in chennai
    rpa training in chennai
    rpa interview questions and answers
    automation anywhere interview questions and answers
    blueprism interview questions and answers
    uipath interview questions and answers
    rpa training in bangalore

    ReplyDelete
  12. Thanks for your informative article, Your post helped me to understand the future and career prospects & Keep on updating your blog with such awesome article.
    Best Devops Training in pune
    Devops interview questions and answers

    ReplyDelete
  13. Whoa! I’m enjoying the template/theme of this website. It’s simple, yet effective. A lot of times it’s very hard to get that “perfect balance” between superb usability and visual appeal. I must say you’ve done a very good job with this.
    aws training in bangalore
    RPA Training in bangalore
    Python Training in bangalore
    Selenium Training in bangalore
    Hadoop Training in bangalore

    ReplyDelete
  14. You are an amazing writer. The content is extra-ordinary. Looking for such a masterpiece. Thanks for sharing.
    IoT Training in Chennai
    IoT courses in Chennai
    IoT Courses
    IoT Training in Porur
    IoT Training in Adyar

    ReplyDelete
  15. Started out in 2012, BIG DATA Central is one of the industry’s leading and fastest growing Internet
    community for data practitioners. Whether it is data science or machine learning or deep learning or
    big data, Data Science Central is a one-stop shop that covers a wide range of data science topics that
    includes technology, tools, data visualisation, code, and job opportunities. Also, many industry experts
    contribute to the community forum for discussion or questions.

    BIGDATATraining in OMR Chennai

    ReplyDelete
  16. A bewildering web journal I visit this blog, it's unfathomably heavenly. Oddly, in this present blog's substance made purpose of actuality and reasonable. The substance of data is informative
    Oracle Fusion Financials Online Training
    Oracle Fusion HCM Online Training
    Oracle Fusion SCM Online Training

    ReplyDelete
  17. Thanks for providing such a useful article containing valuable information. start learning the best online software courses.

    Workday Online Training

    ReplyDelete
  18. Thanks for sharing useful article… Keep on updating such useful information. TN EPayslip | Manidhaneyam IAS Academy Entrance Exam

    ReplyDelete