Hello, Does HBase know how much space it is occupying on HDFS? I looked at these two: http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.html http://hbase.apache.org/book/hbase_metrics.html But I couldn't find any mentions of such a metric. Is this just a matter of exposing this metric? Or...? Thanks, Otis ---- Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm
Hi Otis, This command might be what you want: hadoop fs -dus /hbase -Yifeng On May 7, 2012, at 11:41 AM, Otis Gospodnetic wrote: > Hello, > > Does HBase know how much space it is occupying on HDFS? > I looked at these two: > http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.html > > http://hbase.apache.org/book/hbase_metrics.html > > > But I couldn't find any mentions of such a metric. > > Is this just a matter of exposing this metric? Or...? > > Thanks, > Otis > ---- > Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm >(Continue reading)
You're right, it's not currently a metric. But there is an entry for the disk usage here... http://hbase.apache.org/book.html#trouble.namenode On 5/6/12 10:41 PM, "Otis Gospodnetic" <otis_gospodnetic@...> wrote: >Hello, > >Does HBase know how much space it is occupying on HDFS? >I looked at these two: >http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/metri >cs/RegionServerMetrics.html > >http://hbase.apache.org/book/hbase_metrics.html > > >But I couldn't find any mentions of such a metric. > >Is this just a matter of exposing this metric? Or...? > >Thanks, >Otis >---- >Performance Monitoring for Solr / ElasticSearch / HBase - >http://sematext.com/spm >(Continue reading)
We created our own MetricContext for reading these metrics. Basically your metric context gets called every X seconds based on your hadoop-metrics.properties, so you can add whatever else you want in there. We also were concerned with HDFS usage, and while we couldn't get it to pull that in specifically we did use the java File API to just get the current used disk space for our various mounted drives. This has worked reasonably well, though does not mirror HDFS usage exactly. This is per-server though as opposed to per-table or whatever. You can take a look at the GangliaContext for an example, in fact our MetricContext extends GangliaContext so we can still report to ganglia but also report to our own status system as well. Just put it in a jar, put the jar on the classpath, and reference it in your hadoop-metrics.properties. On Mon, May 7, 2012 at 9:37 AM, Doug Meil <doug.meil@...>wrote: > > You're right, it's not currently a metric. > > But there is an entry for the disk usage here... > > http://hbase.apache.org/book.html#trouble.namenode > > > > > > On 5/6/12 10:41 PM, "Otis Gospodnetic" <otis_gospodnetic@...> wrote:(Continue reading)
Hi Doug, Right, it's not a metric and while hadoop -dus ... is fine from the command line, you can see from my signature why I'd like HBase to expose this metric via JMX. :) Should I open a JIRA issue for this? Thanks, Otis ---- Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm >________________________________ > From: Doug Meil <doug.meil@...> >To: "user <at> hbase.apache.org" <user@...>; Otis Gospodnetic <otis_gospodnetic@...> >Sent: Monday, May 7, 2012 9:37 AM >Subject: Re: HBase HDFS disk space usage > > >You're right, it's not currently a metric. > >But there is an entry for the disk usage here... > >http://hbase.apache.org/book.html#trouble.namenode > > > >(Continue reading)
Jira sounds like a good idea. On 5/7/12 1:52 PM, "Otis Gospodnetic" <otis_gospodnetic@...> wrote: >Hi Doug, > >Right, it's not a metric and while hadoop -dus ... is fine from the >command line, you can see from my signature why I'd like HBase to expose >this metric via JMX. :) > >Should I open a JIRA issue for this? > >Thanks, >Otis >---- > >Performance Monitoring for Solr / ElasticSearch / HBase - >http://sematext.com/spm > > > >>________________________________ >> From: Doug Meil <doug.meil@...> >>To: "user@..." <user@...>; Otis Gospodnetic >><otis_gospodnetic@...> >>Sent: Monday, May 7, 2012 9:37 AM >>Subject: Re: HBase HDFS disk space usage >>(Continue reading)
Done! https://issues.apache.org/jira/browse/HBASE-5960 Otis ---- Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm >________________________________ > From: Doug Meil <doug.meil@...> >To: "user@..." <user <at> hbase.apache.org>; Otis Gospodnetic <otis_gospodnetic@...> >Sent: Monday, May 7, 2012 3:51 PM >Subject: Re: HBase HDFS disk space usage > > >Jira sounds like a good idea. > > > > >On 5/7/12 1:52 PM, "Otis Gospodnetic" <otis_gospodnetic@...> wrote: > >>Hi Doug, >> >>Right, it's not a metric and while hadoop -dus ... is fine from the >>command line, you can see from my signature why I'd like HBase to expose >>this metric via JMX. :) >> >>Should I open a JIRA issue for this?(Continue reading)
RSS Feed