L N | 25 Nov 19:00 2012
Picon

Argumments to Hadoop Command


Hi,

 I am trying to run a hadoop command

 hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input output

 where input and output are directories.But I get the following error.

 WARN mapred.JobClient: Use GenericOptionsParser for parsing the
 arguments. Applications should implement Tool for the same.
 Exception in thread "main"
 org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
 hdfs://.com/user/krishna/input already exists

 If you see it is taking input directory  as output and throwing output
 directory already exists.

 if I give below command

 hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output

 I get below error.

 WARN mapred.JobClient: Use GenericOptionsParser for parsing the
 arguments. Applications should implement Tool for the same.
 Exception in thread "main"
 org.apache.hadoop.mapred.InvalidInputException: Input path does not
 exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>

 Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
 input directory.

 Where am I doing wrong.
 I have implemented tool interface in main class.

I pasted the code in pastbin.com. Link is below.

http://pastebin.com/eN1BXuWh


 Thanks
 Lakshmi

Harsh J | 26 Nov 19:42 2012

Re: Argumments to Hadoop Command

I remember responding with a proper answer earlier, but here it goes
again anyway:

"""
Hi,

Please check http://pastebin.com/phFZgkrQ for a proper version. When
using Tool/Configured, reuse the already prepared configuration (i.e.
getConf()) rather than constructing a new one, otherwise all CLI
parsed options are lost.
"""

On Sun, Nov 25, 2012 at 11:30 PM, L N <nklnarayana@...> wrote:
>
> Hi,
>
>  I am trying to run a hadoop command
>
>  hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
> output
>
>  where input and output are directories.But I get the following error.
>
>  WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>  arguments. Applications should implement Tool for the same.
>  Exception in thread "main"
>  org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
>  hdfs://.com/user/krishna/input already exists
>
>  If you see it is taking input directory  as output and throwing output
>  directory already exists.
>
>  if I give below command
>
>  hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
>  I get below error.
>
>  WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>  arguments. Applications should implement Tool for the same.
>  Exception in thread "main"
>  org.apache.hadoop.mapred.InvalidInputException: Input path does not
>  exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
>  Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
>  input directory.
>
>  Where am I doing wrong.
>  I have implemented tool interface in main class.
>
> I pasted the code in pastbin.com. Link is below.
>
> http://pastebin.com/eN1BXuWh
>
>
>  Thanks
>  Lakshmi
>

--

-- 
Harsh J

Harsh J | 26 Nov 19:42 2012

Re: Argumments to Hadoop Command

I remember responding with a proper answer earlier, but here it goes
again anyway:

"""
Hi,

Please check http://pastebin.com/phFZgkrQ for a proper version. When
using Tool/Configured, reuse the already prepared configuration (i.e.
getConf()) rather than constructing a new one, otherwise all CLI
parsed options are lost.
"""

On Sun, Nov 25, 2012 at 11:30 PM, L N <nklnarayana@...> wrote:
>
> Hi,
>
>  I am trying to run a hadoop command
>
>  hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queue-name> input
> output
>
>  where input and output are directories.But I get the following error.
>
>  WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>  arguments. Applications should implement Tool for the same.
>  Exception in thread "main"
>  org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory
>  hdfs://.com/user/krishna/input already exists
>
>  If you see it is taking input directory  as output and throwing output
>  directory already exists.
>
>  if I give below command
>
>  hadoop jar lc.jar linecount -Dmapred.job.queue.name=<queuename> output
>
>  I get below error.
>
>  WARN mapred.JobClient: Use GenericOptionsParser for parsing the
>  arguments. Applications should implement Tool for the same.
>  Exception in thread "main"
>  org.apache.hadoop.mapred.InvalidInputException: Input path does not
>  exist: hdfs://com/user/krishna/-Dmapred.job.queue.name=<queuename>
>
>  Looks like it is taking -Dmapred.job.queue.name=<queuename> as the
>  input directory.
>
>  Where am I doing wrong.
>  I have implemented tool interface in main class.
>
> I pasted the code in pastbin.com. Link is below.
>
> http://pastebin.com/eN1BXuWh
>
>
>  Thanks
>  Lakshmi
>

--

-- 
Harsh J


Gmane