Pig is a platform for analyzing large datasets and builds on top of Apache Hadoop, an open-source implementation of Google's MapReduce. ()

Vikram Dixit K (JIRA | 23 Jun 2012 04:57

Picon

[Created] (PIG-2766) Pig-HCat Usability

Vikram Dixit K created PIG-2766:
-----------------------------------

             Summary: Pig-HCat Usability
                 Key: PIG-2766
                 URL: https://issues.apache.org/jira/browse/PIG-2766
             Project: Pig
          Issue Type: Bug
          Components: grunt, tools
    Affects Versions: 0.10.0
            Reporter: Vikram Dixit K

Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config
params. We should have better integration between Pig & HCat by pre-configuring Pig to load all these jars
and configs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Vikram Dixit K (JIRA | 23 Jun 2012 05:11

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vikram Dixit K updated PIG-2766:
--------------------------------

    Attachment: PIG-2766.patch

This patch adds a new command line parameter to the bin/pig shell script and includes the appropriate jars
for pig's consumption.

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>         Attachments: PIG-2766.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config
params. We should have better integration between Pig & HCat by pre-configuring Pig to load all these jars
and configs.

(Continue reading)

Vikram Dixit K (JIRA | 23 Jun 2012 05:13

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vikram Dixit K updated PIG-2766:
--------------------------------

    Fix Version/s: 0.10.0
           Status: Patch Available  (was: Open)

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config
params. We should have better integration between Pig & HCat by pre-configuring Pig to load all these jars
and configs.

(Continue reading)

Alan Gates (JIRA | 25 Jun 2012 22:18

Picon

[Assigned] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alan Gates reassigned PIG-2766:
-------------------------------

    Assignee: Vikram Dixit K

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config
params. We should have better integration between Pig & HCat by pre-configuring Pig to load all these jars
and configs.

(Continue reading)

Vikram Dixit K (JIRA | 26 Jun 2012 00:47

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vikram Dixit K updated PIG-2766:
--------------------------------

    Attachment: PIG-2766_2.patch

Added more dependent jars.

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch, PIG-2766_2.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config
params. We should have better integration between Pig & HCat by pre-configuring Pig to load all these jars

(Continue reading)

Daniel Dai (JIRA | 27 Jun 2012 02:47

Picon

[Commented] (PIG-2766) Pig-HCat Usability


    [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401840#comment-13401840
] 

Daniel Dai commented on PIG-2766:
---------------------------------

Couple of comments:
1. We shall not hard code version, instead, find those jars using prefix
2. Why put $additionalJars in the middle? Put it in the end seems more intuitive
3. In the change [[ $f = "-secretDebugCmd" ]], anything wrong with the original syntax?

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch, PIG-2766_2.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set

(Continue reading)

Vikram Dixit K (JIRA | 27 Jun 2012 22:45

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vikram Dixit K updated PIG-2766:
--------------------------------

    Attachment: PIG-2766_3.patch

Updated with comments incorporated.

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch, PIG-2766_2.patch, PIG-2766_3.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config
params. We should have better integration between Pig & HCat by pre-configuring Pig to load all these jars

(Continue reading)

Vikram Dixit K (JIRA | 27 Jun 2012 22:47

Picon

[Commented] (PIG-2766) Pig-HCat Usability


    [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402542#comment-13402542
] 

Vikram Dixit K commented on PIG-2766:
-------------------------------------

Fixed 1 and 2.
3 was just to keep it consistent with the rest of the file.

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch, PIG-2766_2.patch, PIG-2766_3.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config
params. We should have better integration between Pig & HCat by pre-configuring Pig to load all these jars

(Continue reading)

Vikram Dixit K (JIRA | 28 Jun 2012 00:06

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vikram Dixit K updated PIG-2766:
--------------------------------

    Attachment: PIG-2766_4.patch

Removed hive conf from the additional jars list. Added it only to the classpath. 

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch, PIG-2766_2.patch, PIG-2766_3.patch, PIG-2766_4.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config
params. We should have better integration between Pig & HCat by pre-configuring Pig to load all these jars

(Continue reading)

Vikram Dixit K (JIRA | 28 Jun 2012 02:58

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vikram Dixit K updated PIG-2766:
--------------------------------

    Attachment: PIG-2766_5.patch

Added handling for PIG_OPTS route for adding jars.

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch, PIG-2766_2.patch, PIG-2766_3.patch, PIG-2766_4.patch, PIG-2766_5.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config
params. We should have better integration between Pig & HCat by pre-configuring Pig to load all these jars

(Continue reading)

Vikram Dixit K (JIRA | 28 Jun 2012 04:20

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vikram Dixit K updated PIG-2766:
--------------------------------

    Attachment: PIG-2766_6.patch

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch, PIG-2766_2.patch, PIG-2766_3.patch, PIG-2766_4.patch,
PIG-2766_5.patch, PIG-2766_6.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config
params. We should have better integration between Pig & HCat by pre-configuring Pig to load all these jars
and configs.

(Continue reading)

Vikram Dixit K (JIRA | 28 Jun 2012 20:55

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vikram Dixit K updated PIG-2766:
--------------------------------

    Attachment: PIG-2766_7.patch

Moved the pig opts jars to the beginning of the additional jars.

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch, PIG-2766_2.patch, PIG-2766_3.patch, PIG-2766_4.patch,
PIG-2766_5.patch, PIG-2766_6.patch, PIG-2766_7.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config

(Continue reading)

Vikram Dixit K (JIRA | 28 Jun 2012 21:16

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vikram Dixit K updated PIG-2766:
--------------------------------

    Attachment: PIG-2766_Branch0.9.patch

Patch for branch 0.9.

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.10.0
>
>         Attachments: PIG-2766.patch, PIG-2766_2.patch, PIG-2766_3.patch, PIG-2766_4.patch,
PIG-2766_5.patch, PIG-2766_6.patch, PIG-2766_7.patch, PIG-2766_Branch0.9.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set
couple of configuration. For a novice user, it is non-trivial to find all the relevant jars and config

(Continue reading)

Daniel Dai (JIRA | 28 Jun 2012 23:49

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-2766:
----------------------------

       Resolution: Fixed
    Fix Version/s:     (was: 0.10.0)
                   0.10.1
                   0.11
                   0.9.3
     Hadoop Flags: Reviewed
           Status: Resolved  (was: Patch Available)

Patch committed to 0.9/0.10/trunk. Thanks Vikram!

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.9.3, 0.11, 0.10.1
>

(Continue reading)

Vikram Dixit K (JIRA | 29 Jun 2012 09:46

Picon

[Updated] (PIG-2766) Pig-HCat Usability


     [
https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vikram Dixit K updated PIG-2766:
--------------------------------

    Release Note: The patch introduces a new command line parameter for pig: -useHCatalog which imports the
appropriate jars for pig use with HCatalog. This reduces the burden of finding all the relevant jars for
using HCatalog with pig. If the user has setup the home directories for hive/hcatalog those would be used
else defaults (/usr/lib/hive...) are assumed.

> Pig-HCat Usability
> ------------------
>
>                 Key: PIG-2766
>                 URL: https://issues.apache.org/jira/browse/PIG-2766
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, tools
>    Affects Versions: 0.10.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>             Fix For: 0.9.3, 0.11, 0.10.1
>
>         Attachments: PIG-2766.patch, PIG-2766_2.patch, PIG-2766_3.patch, PIG-2766_4.patch,
PIG-2766_5.patch, PIG-2766_6.patch, PIG-2766_7.patch, PIG-2766_Branch0.9.patch
>
>
> Currently to use hcat from pig (via HCatLoader/HCatStorer) user need to register bunch of jars and set

(Continue reading)

Return

Return to gmane.comp.java.hadoop.pig.devel.

Advertisement

Project Web Page

Pig is a platform for analyzing large datasets and builds on top of Apache Hadoop, an open-source implementation of Google's MapReduce. ()

Search Archive

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane