Monday, September 9, 2013

hadoop on cloudera quickstart vm test example 01 wordcount

It's not easy to install hadoop and related items like hdfs hive and so on of your own, and it is more difficult to config them after installation.

Thanks to cloudera, we can test hadoop with its integrated tool kit (Cloudera QuickStart VM). it provides vmware, kvm and virtualbox edition to download. Everything is configured and you can test without any difficulty.

in this video, I show a example hot to do wordcount in hadoop. The youtube link is:


Steps not included in the video:
1: download vmware player or virtualbox and install;
2: download Cloudera QuickStart VM from cloudera(the link may change all the time, so you can google keyword "Cloudera QuickStart VM" to download).


The example of the wordcount test is:

1: install wget on centos server
sudo yum -y install wget
2: create test dir in /home/cloudera
mkdir /home/cloudera/test
cd /home/cloudera/test
3: create test txt file
echo "what can I do with hadoop on hadoop server or hive server" > test1.txt
4: put the txt file to hdfs
hdfs dfs -mkdir /user/cloudera/input
hdfs dfs -put /home/cloudera/test/test1.txt /user/cloudera/input/
5: go to  /usr/lib/hadoop-mapreduce/
cd /usr/lib/hadoop-mapreduce/
6: run the job
hadoop jar hadoop-mapreduce-examples.jar wordcount /user/cloudera/input/test1.txt /user/cloudera/output
7: check what are there in the output
hdfs dfs -ls /user/cloudera/output/
8: reat the output file
hdfs dfs -cat /user/cloudera/output/part-r-00000

75 comments:

  1. This is one of the most incredible blogs on hadoop. Ive read in a very long time. The amount of information in here is stunning, like you practically wrote the book on the subject. Your blog is great for anyone who wants to understand this subject more. Great stuff; please keep it up!
    Hadoop Training in hyderabad

    ReplyDelete
  2. Actually, you have explained the technology to the fullest. Thanks for sharing the information you have got. It helped me a lot. I experimented your thoughts in my training program.



    Hadoop Training Chennai
    Hadoop Training in Chennai
    Big Data Training in Chennai

    ReplyDelete
  3. Cloud is one of the tremendous technology that any company in this world would rely on(cloud computing training chennai). Using this technology many tough tasks can be accomplished easily in no time. Your content are also explaining the same(Cloud computing training centers in chennai). Thanks for sharing this in here. You are running a great blog, keep up this good work.

    ReplyDelete
  4. Well Said. The content provided is true up to my knowledge. This made me to understand the concepts very clear. Thanks for sharing this wonderful information in here. Keep blogging article like this. I have bookmarked this page for future reference as well.


    Hadoop Training Chennai | Big Data Training | JAVA Course in Chennai

    ReplyDelete
  5. Thank you a lot for providing individuals with a very spectacular possibility to read critical reviews from this site.

    Best Java Training Institute Chennai

    ReplyDelete
  6. I am happy to this post..Interesting post! Thanks for writing it.What's wrong with this kind of post exactly? It follows your previous guideline for post length as well as clarity.aws vpc interview questions

    ReplyDelete
  7. Thanks a lot very much for the high quality and results-oriented help. I won’t think twice to endorse your blog post to anybody who wants and needs support about this area.
    digital marketing training in marathahalli

    digital marketing training in rajajinagar

    Digital Marketing Training in online


    full stack developer training in pune


    full stack developer training in annanagar

    ReplyDelete
  8. Existing without the answers to the difficulties you’ve sorted out through this guide is a critical case, as well as the kind which could have badly affected my entire career if I had not discovered your website.
    full stack developer training in tambaram

    full stack developer training in velachery



    ReplyDelete
  9. Nice Blog, When i was read this blog i learnt new things & its truly have well stuff related to developing technology, Thank you for sharing this blog. Digital Marketing Training in Mumbai

    ReplyDelete
  10. It’s great to come across a blog every once in a while that isn’t the same out of date rehashed material. Fantastic read.

    Digital Marketing Training in Mumbai

    Six Sigma Training in Dubai

    Six Sigma Abu Dhabi

    ReplyDelete
  11. Great post! I am actually getting ready to across this information, It’s very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.
    Python training in btm
    Python training in marathahalli
    AWS Training in chennai

    ReplyDelete
  12. I think you have a long story to share and i am glad after long time finally you cam and shared your experience.
    Blueprism training in Pune

    Blueprism online training

    Blue Prism Training in Pune

    ReplyDelete
  13. Thanks for the informative article. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.
    Data Science Training in Chennai
    Data science training in bangalore
    Data science online training
    Data science training in pune

    ReplyDelete
  14. This blog gives very important info about SAS Thanks for sharing.Very well written information. Many

    thanks.
    SAS training

    ReplyDelete
  15. Very nice post here and thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.
    Good discussion. Thank you.
    Anexas
    Six Sigma Training in Abu Dhabi
    Six Sigma Training in Dammam
    Six Sigma Training in Riyadh

    ReplyDelete
  16. Very nice post here and thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.
    java training in chennai | java training in bangalore

    java online training | java training in pune

    ReplyDelete
  17. The knowledge of technology you have been sharing thorough this post is very much helpful to develop new idea. here by i also want to share this.

    angularjs-Training in velachery

    angularjs Training in bangalore

    angularjs Training in bangalore

    angularjs Training in btm

    angularjs Training in electronic-city

    ReplyDelete
  18. I simply wanted to thank you so much again. I am not sure the things that I might have gone through without the type of hints revealed by you regarding that situation.
    safety course in chennai

    ReplyDelete
  19. Appreciation for really being thoughtful and also for deciding on certain marvelous guides most people really want to be aware of.

    Cloud Training
    Software Testing Training
    Tableau Training in Chennai
    QlikView Training in Chennai
    Microstrategy Training in Chennai

    ReplyDelete
  20. Very interesting information that you have shared with us.i have personally thank you for sharing your ideas with us.
    android development course in bangalore
    Android Training in Thirumangalam
    Android Training in Amjikarai
    Android Training in Padur

    ReplyDelete
  21. This blog is full of Innovative ideas.surely i will look into this insight.please add more information's like this soon.
    AWS Course in Anna Nagar
    Best AWS Training Institute in Anna nagar
    AWS Courses in T nagar
    AWS Training Institutes in T nagar

    ReplyDelete
  22. I am really enjoying reading your well-written articles. It looks like you spend a lot of effort and time on your blog. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work.
    RPA Training in Chennai
    RPA Training Institute in Chennai
    Robotic Process Automation Training
    Robotics courses in bangalore
    RPA classes in bangalore

    ReplyDelete
  23. Do you mind if I quote a couple of your posts as long as I provide credit and sources back to your blog?
    fire and safety course in chennai

    ReplyDelete
  24. Amazing article. Your blog helped me to improve myself in many ways thanks for sharing this kind of wonderful informative blogs in live. I have bookmarked more article from this website. Such a nice blog you are providing ! Kindly Visit Us @ Tours and Travels in Madurai | Best Travels in Madurai | Madurai Travels

    ReplyDelete
  25. Nice tips. Very innovative... Your post shows all your effort and great experience towards your work Your Information is Great if mastered very well.
    Microsoft Azure online training
    Selenium online training
    Java online training
    Java Script online training
    Share Point online training

    ReplyDelete
  26. Thanks for splitting your comprehension with us. It’s really useful to me & I hope it helps the people who in need of this vital information. 
    devops online training

    aws online training

    data science with python online training

    data science online training

    rpa online training

    ReplyDelete

  27. Thank you for sharing the article. The data that you provided in the blog is informative and

    effective. Best Devops Training Institute

    ReplyDelete
  28. Thanks for posting this highly informative article. I feel glad about learning more about this concept. Maintain the number of posting and keep up the good work!
    Data Science Course in Chennai | Data Science Training in Chennai

    ReplyDelete
  29. Thanks for posting this highly informative article. I feel glad about learning more about this concept. Maintain the number of posting and keep up the good work!
    Data Science Course in Chennai | Data Science Training in Chennai

    ReplyDelete
  30. Hey Nice Blog!! Thanks For Sharing!!! Wonderful blog & good post. It is really very helpful to me, waiting for a more new post. Keep Blogging ! Here is the best angular training with free Bundle videos .

    contact No :- 9885022027.
    SVR Technologies


    ReplyDelete
  31. That is a very good tip especially to those fresh to the blogosphere. Brief but very precise info… Thank you for sharing this one. A must read website article!

    ReplyDelete
  32. Wow! this is Amazing! Do you know your hidden name meaning ? Click here to find your hidden name meaning

    ReplyDelete
  33. Attend The Business Analytics Courses From ExcelR. Practical Business Analytics Courses Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analytics Courses.
    ExcelR Business Analytics Courses
    Data Science Interview Questions

    ReplyDelete
  34. Nice blog,I understood the topic very clearly,And want to study more like this.
    Data Scientist Course

    ReplyDelete
  35. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful.

    Digital marketing course

    ReplyDelete
  36. Inspirational Blog The Content Mentioned is Knowledgeable and Very Effective .Thanks For Sharing It

    AI Training In Hyderabad

    ReplyDelete
  37. This is what I’ve been looking for. Had a lot of doubts regarding Cloud computing. Thanks for sharing nice information.
    Azure Training in Hyderabad

    ReplyDelete
  38. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.
    Machine Learning Training In Hyderabad

    Machine Learning Course In Hyderabad

    ReplyDelete
  39. Very informative post. Social media will give the loads of opportunities to grow your business and helps you attract the targeted audiences (both B2B and B2C). Sharing more infographic content on social media will give more visibility.Thanks for sharing...keep update...
    Very informative post. Social media will give the loads of opportunities to grow your business and helps you attract the targeted audiences (both B2B and B2C). Sharing more infographic content on social media will give more visibility.Thanks for sharing...keep update...
    very nice to read this page
    Ai & Artificial Intelligence Course in Chennai
    PHP Training in Chennai
    Ethical Hacking Course in Chennai Blue Prism Training in Chennai
    UiPath Training in Chennai

    ReplyDelete
  40. The context has been explained really well. Looking forward to see more of such informative updates
    Machine Learning Training in Hyderabad

    ReplyDelete
  41. This article is well formulated. I particularly like the way how you have delivered all the major points about the topic of the content in petite and crisp points.

    Data Science training in Mumbai
    Data Science course in Mumbai
    SAP training in Mumbai

    ReplyDelete
  42. Attend The Data Analyst Course From ExcelR. Practical Data Analyst Course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analyst Course.
    Data Analyst Course

    ReplyDelete
  43. Good Post! , it was so good to read and useful to improve my knowledge as an updated one, keep blogging.After seeing your article I want to say that also a well-written article with some very good information which is very useful for the readers....thanks for sharing it and do share more posts likethis. https://www.3ritechnologies.com/course/tableau-online-training-certification/

    ReplyDelete
  44. This comment has been removed by the author.

    ReplyDelete
  45. Very good information which is very useful for the readers....thanks for sharing it and do share more posts like this.

    Data Science Training in Gurgaon
    Bigdata Hadoop Training in Gurgaon

    ReplyDelete
  46. wonderful article contains lot of valuable information. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.
    This article resolved my all queries.good luck an best wishes to the team members.learn digital marketing use these following link
    Digital Marketing Course in Chennai

    ReplyDelete
  47. Attend The Course in Data Analytics From ExcelR. Practical Course in Data Analytics Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Course in Data Analytics.
    Course in Data Analytics

    ReplyDelete
  48. Thanks for posting the best information and the blog is very helpful.data science interview questions and answers

    ReplyDelete
  49. cyber security training london- Leading CompTIA Cybersecurity Analyst CySA+ Training provider in london. We provide all CompTIA courses. 100% Hands on practical.

    ReplyDelete


  50. I was just examining through the web looking for certain information and ran over your blog.It shows how well you understand this subject. Bookmarked this page, will return for extra. data science course in vadodara

    ReplyDelete
  51. Extremely overall quite fascinating post. I was searching for this sort of data and delighted in perusing this one. Continue posting. A debt of gratitude is in order for sharing. python course in delhi

    ReplyDelete