{"id":1257,"date":"2016-06-12T12:57:49","date_gmt":"2016-06-12T20:57:49","guid":{"rendered":"http:\/\/www.developerscloset.com\/?page_id=1257"},"modified":"2018-06-14T09:44:57","modified_gmt":"2018-06-14T17:44:57","slug":"opentsdb-timeseries-database","status":"publish","type":"page","link":"https:\/\/www.developerscloset.com\/?page_id=1257","title":{"rendered":"OpenTSDB: Timeseries Database"},"content":{"rendered":"<p><a href=\"http:\/\/www.developerscloset.com\/wp-content\/uploads\/2018\/06\/opentsdb-logo-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-medium wp-image-1258 alignnone\" src=\"http:\/\/www.developerscloset.com\/wp-content\/uploads\/2018\/06\/opentsdb-logo-1-300x62.png\" alt=\"\" width=\"300\" height=\"62\" srcset=\"https:\/\/www.developerscloset.com\/wp-content\/uploads\/2018\/06\/opentsdb-logo-1-300x62.png 300w, https:\/\/www.developerscloset.com\/wp-content\/uploads\/2018\/06\/opentsdb-logo-1.png 364w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>To support the collection of various time series data for dashboarding\/alerting purposes, I recommend OpenTSDB.\u00a0OpenTSDB is a time series database built on top of HBase\u00a0<a class=\"external-link\" href=\"http:\/\/opentsdb.net\/\" rel=\"nofollow\">http:\/\/opentsdb.net\/<\/a><\/p>\n<div class=\"toc-macro client-side-toc-macro conf-macro output-block hidden-outline\">\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_79 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-69ea2338b9526\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-69ea2338b9526\"  aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.developerscloset.com\/?page_id=1257\/#Installation\" >Installation<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.developerscloset.com\/?page_id=1257\/#OpenTSDB_Setup_instructions\" >OpenTSDB Setup instructions<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.developerscloset.com\/?page_id=1257\/#How_to_Use_OpenTSDB\" >How to Use OpenTSDB<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.developerscloset.com\/?page_id=1257\/#Writing_data_to_OpenTSDB\" >Writing data to OpenTSDB<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.developerscloset.com\/?page_id=1257\/#Querying_Data_from_OpenTSDB\" >Querying Data from OpenTSDB<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.developerscloset.com\/?page_id=1257\/#References\" >References<\/a><\/li><\/ul><\/nav><\/div>\n<h1 id=\"OpenTSDB:TimeseriesDatabase-Installation\"><span class=\"ez-toc-section\" id=\"Installation\"><\/span>Installation<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<h2 id=\"OpenTSDB:TimeseriesDatabase-OpenTSDBSetupinstructions\"><span class=\"ez-toc-section\" id=\"OpenTSDB_Setup_instructions\"><\/span>OpenTSDB Setup instructions<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Setting up test environment on your Linux VM:<\/p>\n<ol>\n<li>Setup an HBase cluster on your Dev machine (can be done through Cloudera Manager\/CDH, test that it is running by visiting in a web browser: http:\/\/&lt;yourdevmachine&gt;:60010<\/li>\n<li>Obtain the Debian package for opentsdb from their releases page:\n<ol>\n<li><a class=\"external-link\" href=\"https:\/\/github.com\/OpenTSDB\/opentsdb\/releases\" rel=\"nofollow\">https:\/\/github.com\/OpenTSDB\/opentsdb\/releases<\/a><\/li>\n<li>you can download the .deb file by running: wget\u00a0<a class=\"external-link\" href=\"https:\/\/github.com\/OpenTSDB\/opentsdb\/releases\/download\/v2.2.0\/opentsdb-2.2.0_all.deb\" rel=\"nofollow\">https:\/\/github.com\/OpenTSDB\/opentsdb\/releases\/download\/v2.2.0\/opentsdb-2.2.0_all.deb<\/a><\/li>\n<\/ol>\n<\/li>\n<li>Install the debian package by running: sudo dpkg -i opentsdb-2.2.0_all.deb<\/li>\n<li>Install other dependency, gnuplot: sudo apt-get install -y gnuplot<\/li>\n<li>Create the opentsdb tables in hbase by running:\n<ol>\n<li>env COMPRESSION=SNAPPY HBASE_HOME=\/opt\/cloudera\/parcels\/CDH\/lib\/hbase \/usr\/share\/opentsdb\/tools\/create_table.sh<\/li>\n<li>This command does not require root or sudo<\/li>\n<\/ol>\n<\/li>\n<li>Add following configuration to the opentsdb config file at \/etc\/opentsdb\/opentsdb.conf: tsd.core.auto_create_metrics = true<\/li>\n<li>Start the TSDB daemon by running: sudo service opentsdb start<\/li>\n<li>Verify and test\n<ol>\n<li>logs are stored in \/var\/log\/opentsdb\/opentsdb.log<\/li>\n<li>you can view the basic ui at http:\/\/&lt;yourdevmachine&gt;:4242<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<h1 id=\"OpenTSDB:TimeseriesDatabase-HowtoUseOpenTSDB\"><span class=\"ez-toc-section\" id=\"How_to_Use_OpenTSDB\"><\/span>How to Use OpenTSDB<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<h2 id=\"OpenTSDB:TimeseriesDatabase-WritingdatatoOpenTSDB\"><span class=\"ez-toc-section\" id=\"Writing_data_to_OpenTSDB\"><\/span>Writing data to OpenTSDB<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Writing data to OpenTSDB can be done either through its REST API or via direct socket connections. The general message written to TSDB requires the following information:<\/p>\n<ul>\n<li>metric name\n<ul>\n<li>Namespacing of metrics is important to be able to quickly query data in the way you expect viewers of the data would want to query it.<\/li>\n<li>Example name:\u00a0webservers.sys.cpu.user.all<\/li>\n<li>Case sensitive<\/li>\n<li>No spaces in name allowed<\/li>\n<li>Allowed chars:\u00a0<code class=\"docutils literal\"><span class=\"pre\">a<\/span><\/code>\u00a0to\u00a0<code class=\"docutils literal\"><span class=\"pre\">z<\/span><\/code>,\u00a0<code class=\"docutils literal\"><span class=\"pre\">A<\/span><\/code>\u00a0to\u00a0<code class=\"docutils literal\"><span class=\"pre\">Z<\/span><\/code>,\u00a0<code class=\"docutils literal\"><span class=\"pre\">0<\/span><\/code>\u00a0to\u00a0<code class=\"docutils literal\"><span class=\"pre\">9<\/span><\/code>,\u00a0<code class=\"docutils literal\"><span class=\"pre\">-<\/span><\/code>,\u00a0<code class=\"docutils literal\"><span class=\"pre\">_<\/span><\/code>,\u00a0<code class=\"docutils literal\"><span class=\"pre\">.<\/span><\/code>,\u00a0<code class=\"docutils literal\"><span class=\"pre\">\/<\/span><\/code><\/li>\n<li>Important thought must be decided up front about how to name the time series because it serves as the default aggregation unit when no tags are specified in queries\n<ul>\n<li>Example: If you care about the total size of a kafka topic, you can still gather metrics for each partition but you would identify each partition via a tag and use the same metric name for all partitions<\/li>\n<li>&#8220;kafka.topic.mykafkatopic.size 1466017245 100\u00a0partition=0&#8221;<\/li>\n<li>&#8220;kafka.topic.mykafkatopic.size\u00a01466017245 150 partition=1&#8221;<\/li>\n<li>&#8220;kafka.topic.mykafkatopic.size\u00a01466017245 200 partition=2&#8221;<\/li>\n<li>A query for kafka.topic.mykafkatopic.size would automatically sum each of the partitions to produce a total size across all partitions, which is the most common query for this metric anyway<\/li>\n<li>You can still query for each partition individually by adding the tag &#8220;partition=0&#8221; to the query<\/li>\n<\/ul>\n<\/li>\n<li>More details at:\u00a0<a class=\"external-link\" href=\"http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/writing.html#naming-schema\" rel=\"nofollow\">http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/writing.html#naming-schema<\/a><\/li>\n<\/ul>\n<\/li>\n<li>timestamp\n<ul>\n<li>Value of seconds or milliseconds since epoch<\/li>\n<li>generally should stick to seconds of precision<\/li>\n<li>More details at:\u00a0<a class=\"external-link\" href=\"http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/writing.html#timestamps\" rel=\"nofollow\">http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/writing.html#timestamps<\/a><\/li>\n<\/ul>\n<\/li>\n<li>value\n<ul>\n<li>TSDB currently supports numeric values only<\/li>\n<li>Can be int or float<\/li>\n<li>More details at:\u00a0<a class=\"external-link\" href=\"http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/writing.html#integer-values\" rel=\"nofollow\">http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/writing.html#integer-values<\/a><\/li>\n<\/ul>\n<\/li>\n<li>list of tags\n<ul>\n<li>Tags are possible dimensions to slice a particular metric name<\/li>\n<li>Each metric must have at least 1 tag<\/li>\n<li>Recommend to use a host=&lt;hostname&gt; tag for the metric<\/li>\n<li>Tags gives flexibility to query metrics but too many will be a performance hit<\/li>\n<li>Recommend to keep number of tags per metric to be less than 8<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Send data via REST API example<\/p>\n<ul>\n<li>See the tcollector main class or details at\u00a0<a class=\"external-link\" href=\"http:\/\/opentsdb.net\/docs\/build\/html\/api_http\/put.html\" rel=\"nofollow\">http:\/\/opentsdb.net\/docs\/build\/html\/api_http\/put.html<\/a><\/li>\n<\/ul>\n<p>Send data via simple socket example<\/p>\n<ul>\n<li>General example: put &lt;metric&gt; &lt;timestamp&gt; &lt;value&gt; &lt;tagk1=tagv1[ tagk2=tagv2 &#8230;tagkN=tagvN]&gt;<\/li>\n<li>CLI example: echo &#8220;put\u00a0kafka.topic.mykafkatopic.size\u00a01466017245 200 partition=2&#8221; | nc &lt;tsd host&gt; 4242\n<ul>\n<li>This pipes the echo output over a simple tcp socket to the tsdb server<\/li>\n<li>Can only send one metric at a time<\/li>\n<li>Must include a newline at the end (which echo does by default)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Installing tcollector &#8211;\u00a0Sending your dev vm&#8217;s system metrics to your local OpenTSDB instance to have some test data to work with<\/p>\n<ol>\n<li>Obtain the latest version at\u00a0<a class=\"external-link\" href=\"https:\/\/github.com\/OpenTSDB\/tcollector\/releases\" rel=\"nofollow\">https:\/\/github.com\/OpenTSDB\/tcollector\/releases<\/a>\n<ol>\n<li>wget\u00a0<a class=\"external-link\" href=\"https:\/\/github.com\/OpenTSDB\/tcollector\/archive\/v1.3.1.tar.gz\" rel=\"nofollow\">https:\/\/github.com\/OpenTSDB\/tcollector\/archive\/v1.3.1.tar.gz<\/a><\/li>\n<\/ol>\n<\/li>\n<li>untar to directory of choice\n<ol>\n<li>(untar to \/opt\/tcollector-v1.3.1) sudo tar -xzf v1.3.1.tar.gz -d \/opt\/<\/li>\n<\/ol>\n<\/li>\n<li>run sudo .\/tcollector start to start the collector daemon<\/li>\n<li>logs to \/var\/log\/tcollector.log<\/li>\n<li>by default, will start all collectors that match running services on your system\n<ol>\n<li>collectors are installed under \/opt\/tcollector-v1.3.1\/collectors<\/li>\n<li>can manually modify which collectors to run by enabling the execute bit on the collectors<\/li>\n<\/ol>\n<\/li>\n<li>start|stop the tcollectors by running sudo .\/tcollector &lt;action&gt;<\/li>\n<li>Some collectors require configuration parameters under \/opt\/tcollector-v1.3.1\/collectors\/etc\/<\/li>\n<li>Additional documentation:\u00a0<a class=\"external-link\" href=\"http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/utilities\/tcollector.html\" rel=\"nofollow\">http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/utilities\/tcollector.html<\/a><\/li>\n<\/ol>\n<p>The included collectors with the tcollector package provide a lot of examples to send data, all written in Python.\u00a0More details at:\u00a0<a class=\"external-link\" href=\"http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/writing.html#input-methods\" rel=\"nofollow\">http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/writing.html#input-methods<\/a><\/p>\n<h2 id=\"OpenTSDB:TimeseriesDatabase-QueryingDatafromOpenTSDB\"><span class=\"ez-toc-section\" id=\"Querying_Data_from_OpenTSDB\"><\/span>Querying Data from OpenTSDB<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Basic frontend is available via http on the same port as the write port (default 4242). Metric names can be searched with autocomplete suggestions available to help find metrics. For dashboard purposes, a frontend like Grafana makes it a little easier to query TSDB but the basics of knowing how to query metric names, setting tags, and downsampling is important.<\/p>\n<p>OpenTSDB supports very high precision from older metric systems, down to the seconds. This power allows for querying at very high resolution but if we&#8217;re looking at large windows, that might be more data points than we need. Rather than generating rollups and discarding data, it downsamples data. Downsampling is simply picking an aggregation function (sum, avg, count, max, min, etc) over the downsampling window. For example, if I query for 7 days worth of datapoints for a metric that we collect every 15 seconds, we might want to downsample to the nearest 5 minute or higher so that we&#8217;re not overwhelmed by the number of data points. This feature is optional so you can always drill down to the lowest level still.<\/p>\n<p>More info about the theory of querying OpenTSDB available at:\u00a0<a class=\"external-link\" href=\"http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/query\/aggregators.html\" rel=\"nofollow\">http:\/\/opentsdb.net\/docs\/build\/html\/user_guide\/query\/aggregators.html<\/a><\/p>\n<p>In general, we&#8217;ll be querying data for either displaying in dashboards or programmatically for individual datapoints for alerting. Both cases uses the OpenTSDB HTTP API to retrieve data.<\/p>\n<p>Example query to retrieve the last value of a metric for threshold checking<\/p>\n<div class=\"code panel pdl conf-macro output-block\">\n<div class=\"codeHeader panelHeader pdl hide-border-bottom\"><b class=\" code-title\">Example Python OpenTSDB REST query<\/b><span class=\"collapse-source expand-control\"><span class=\"expand-control-icon icon\">\u00a0<\/span><span class=\"expand-control-text\">Expand source<\/span><\/span><\/div>\n<\/div>\n<p>More info at:\u00a0<a class=\"external-link\" href=\"http:\/\/opentsdb.net\/docs\/build\/html\/api_http\/index.html\" rel=\"nofollow\">http:\/\/opentsdb.net\/docs\/build\/html\/api_http\/index.html<\/a><\/p>\n<h1 id=\"OpenTSDB:TimeseriesDatabase-Additionaldocumentation\"><span class=\"ez-toc-section\" id=\"References\"><\/span>References<span class=\"ez-toc-section-end\"><\/span><\/h1>\n<p>More info at\u00a0<a class=\"external-link\" href=\"http:\/\/opentsdb.net\/docs\/build\/html\/index.html\" rel=\"nofollow\">http:\/\/opentsdb.net\/docs\/build\/html\/index.html<\/a><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>To support the collection of various time series data for dashboarding\/alerting purposes, I recommend OpenTSDB.\u00a0OpenTSDB is a time series database built on top of HBase\u00a0http:\/\/opentsdb.net\/ [&#8230;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"class_list":["post-1257","page","type-page","status-publish","hentry"],"jetpack_shortlink":"https:\/\/wp.me\/P1BQ8S-kh","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/pages\/1257","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1257"}],"version-history":[{"count":3,"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/pages\/1257\/revisions"}],"predecessor-version":[{"id":1424,"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=\/wp\/v2\/pages\/1257\/revisions\/1424"}],"wp:attachment":[{"href":"https:\/\/www.developerscloset.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1257"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}