hadoop - Error in the code. pig language -



hadoop - Error in the code. pig language -

hello programmers okay?

i programming , run pig command returns me syntax error in code , unable solve.

below post code , error raised.

apparently error occurs in syntax of line 41 sortedresults

code:

--load files relations month1 = load 'hdfs:/data/big/data/weather/201201hourly.txt' using pigstorage(','); month2 = load 'hdfs:/data/big/data/weather/201202hourly.txt' using pigstorage(','); month3 = load 'hdfs:/data/big/data/weather/201203hourly.txt' using pigstorage(','); month4 = load 'hdfs:/data/big/data/weather/201204hourly.txt' using pigstorage(','); month5 = load 'hdfs:/data/big/data/weather/201205hourly.txt' using pigstorage(','); month6 = load 'hdfs:/data/big/data/weather/201206hourly.txt' using pigstorage(','); --combine relations months = union month1, month2, month3, month4, month5, month6; /* splitting relations split months splitmonth1 if substring(date, 4, 6) == '01', splitmonth2 if substring(date, 4, 6) == '02', splitmonth3 if substring(date, 4, 6) == '03', splitrest if (substring(date, 4, 6) == '04' or substring(date, 4, 6) == '04'); */ /* joining relations stations = load 'hdfs:/data/big/data/qclcd201211/stations.txt' using pigstorage() (id:int, name:chararray) bring together months wban, stations id; */ --filter out unwanted info clearweather = filter months skycondition == 'clr'; --transform , shape relation shapedweather = foreach clearweather generate date, substring(date, 0, 4) year, substring(date, 4, 6) month, substring(date, 6, 8) day, skycondition, drytemp; --group relation specifying number of reducers groupedmonthday = grouping shapedweather (month, day) parallel 10; --aggregate relation aggedresults = foreach groupedbymonthday generate grouping monthday, avg(shapedweather.drytemp), min(shapedweather.drytemp), max(shapedweather.drytemp), count(shapedweather.drytemp) parallel 10; --sort relation sortedresults = sort aggedresults $1 desc; --store results in hdfs store sortedresults 'hdfs:/data/big/data/weather/pigresults' using pigstorage(':');

here homecoming play file contained in code.

erro:

14/10/20 22:30:47 warn pig.main: cannot write log file: //pig_1413851447162.log 2014-10-20 22:30:47,165 [main] info org.apache.pig.main - apache pig version 0.12.1 (r1585011) compiled apr 05 2014, 01:41:34 2014-10-20 22:30:47,486 [main] info org.apache.pig.impl.util.utils - default bootup file /home/eduardo/.pigbootup not found 2014-10-20 22:30:47,510 [main] warn org.apache.pig.main - cannot write log file: ///weather.pig1413851447510.log 2014-10-20 22:30:47,629 [main] info org.apache.pig.backend.hadoop.executionengine.hexecutionengine - connecting hadoop file scheme at: hdfs://localhost:54310 2014-10-20 22:30:47,770 [main] info org.apache.pig.backend.hadoop.executionengine.hexecutionengine - connecting map-reduce job tracker at: localhost:54311 2014-10-20 22:30:48,005 [main] error org.apache.pig.pigserver - exception during parsing: error during parsing. <file home/eduardo/documentos/pig/weather.pig, line 41, column 0> syntax error, unexpected symbol @ or near 'sortedresults' failed parse: <file home/eduardo/documentos/pig/weather.pig, line 41, column 0> syntax error, unexpected symbol @ or near 'sortedresults' @ org.apache.pig.parser.queryparserdriver.parse(queryparserdriver.java:241) @ org.apache.pig.parser.queryparserdriver.parse(queryparserdriver.java:179) @ org.apache.pig.pigserver$graph.parsequery(pigserver.java:1678) @ org.apache.pig.pigserver$graph.access$000(pigserver.java:1411) @ org.apache.pig.pigserver.parseandbuild(pigserver.java:344) @ org.apache.pig.pigserver.executebatch(pigserver.java:369) @ org.apache.pig.pigserver.executebatch(pigserver.java:355) @ org.apache.pig.tools.grunt.gruntparser.executebatch(gruntparser.java:140) @ org.apache.pig.tools.grunt.gruntparser.parsestoponerror(gruntparser.java:202) @ org.apache.pig.tools.grunt.gruntparser.parsestoponerror(gruntparser.java:173) @ org.apache.pig.tools.grunt.grunt.exec(grunt.java:84) @ org.apache.pig.main.run(main.java:607) @ org.apache.pig.main.main(main.java:156) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:57) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43) @ java.lang.reflect.method.invoke(method.java:606) @ org.apache.hadoop.util.runjar.main(runjar.java:160) 2014-10-20 22:30:48,008 [main] error org.apache.pig.tools.grunt.grunt - error 1200: <file home/eduardo/documentos/pig/weather.pig, line 41, column 0> syntax error, unexpected symbol @ or near 'sortedresults' 2014-10-20 22:30:48,008 [main] warn org.apache.pig.tools.grunt.grunt - there no log file write to. 2014-10-20 22:30:48,008 [main] error org.apache.pig.tools.grunt.grunt - org.apache.pig.impl.logicallayer.frontendexception: error 1000: error during parsing. <file home/eduardo/documentos/pig/weather.pig, line 41, column 0> syntax error, unexpected symbol @ or near 'sortedresults' @ org.apache.pig.pigserver$graph.parsequery(pigserver.java:1691) @ org.apache.pig.pigserver$graph.access$000(pigserver.java:1411) @ org.apache.pig.pigserver.parseandbuild(pigserver.java:344) @ org.apache.pig.pigserver.executebatch(pigserver.java:369) @ org.apache.pig.pigserver.executebatch(pigserver.java:355) @ org.apache.pig.tools.grunt.gruntparser.executebatch(gruntparser.java:140) @ org.apache.pig.tools.grunt.gruntparser.parsestoponerror(gruntparser.java:202) @ org.apache.pig.tools.grunt.gruntparser.parsestoponerror(gruntparser.java:173) @ org.apache.pig.tools.grunt.grunt.exec(grunt.java:84) @ org.apache.pig.main.run(main.java:607) @ org.apache.pig.main.main(main.java:156) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:57) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43) @ java.lang.reflect.method.invoke(method.java:606) @ org.apache.hadoop.util.runjar.main(runjar.java:160) caused by: failed parse: <file home/eduardo/documentos/pig/weather.pig, line 41, column 0> syntax error, unexpected symbol @ or near 'sortedresults' @ org.apache.pig.parser.queryparserdriver.parse(queryparserdriver.java:241) @ org.apache.pig.parser.queryparserdriver.parse(queryparserdriver.java:179) @ org.apache.pig.pigserver$graph.parsequery(pigserver.java:1678) ... 15 more

pig doesn't have sort command. have seek "order" command. seek this

sortedresults = order aggedresults $1 desc;

hadoop apache-pig hadoop-plugins

Comments

Post a Comment

Popular posts from this blog

xslt - DocBook 5 to PDF transform failing with error: "fo:flow" is missing child elements. Required content model: marker* -

xslt - DocBook 5 to PDF transform failing with error: "fo:flow" is missing child elements. Required content model: marker* - i've inherited doc publishing process takes docbook , builds html , pdf output, using apache fop both. project had been started, never completed. lots of tweaking, i've able of doc sets build (20+), except handful, , fail severe code 1 of these 2 conditions: "fo:flow" missing kid elements. required content model: marker* "fo:block" not valid kid of "fo:root" the xml output comes out of docbook , docbook-fop checks out well-formed. the fop error in 1 book, "endecagloss", example, points end of string: <fo:flow flow-name="xsl-region-body" start-indent="4pc" end-indent="0pt"/> here’s log output in case: [java] severe: javax.xml.transform.transformerexception: file:/scratch/publishing/hudson/jobs/build-main-endeca-documentation/workspace/serverdoc/w...
Read more

mediawiki - How do I insert tables inside infoboxes on Wikia pages? -

mediawiki - How do I insert tables inside infoboxes on Wikia pages? - i want insert tables in infoboxes. example, have proteinbox i've developed using infobox template (http://health-and-medicine.wikia.com/wiki/template:proteinbox), , fasta field big fit comfortably in proteinbox. hence collapsible table within box. this tried: {{infobox | row 1 = {| class = "mw-collapsible mw-collapsed wikitable" |- | <!-- table content --> |} }} your problem is, putting nested table within template. pipe characters ( | ) in table syntax collides pipe usage in templates. the commonly used hack around create template called template:! , containing pipe character, , utilize when need set tables, parser functions or other stuff using pipe characters, within templates. table (with every | replaced {{!}} ): {{{!}} {{!}}- {{!}} a1 {{!}} b1 {{!}}- {{!}} a2 {{!}} b2 {{!}}} ...the equivalent of {| |- | a1 | b1 |- | a2 | b2 |} furthermore, hav...
Read more

Local Service User Logged into Windows -

Local Service User Logged into Windows - i created windows service tracks app server user connected through load balancer. it retrieves user logged windows on machine, cross references active directory (ad) business relationship homecoming first name, lastly name , e-mail (for notification through website created project) there 1 scheme on network users remote system, vista system, reason returning values "local service" logged machine, it's weird , security guys impossible "local service" business relationship logged windows. here how user logged windows managementscope ms = new managementscope("\\\\.\\root\\cimv2"); objectquery query = new objectquery("select * win32_computersystem"); managementobjectsearcher searcher = new managementobjectsearcher(ms, query); foreach (managementobject mo in searcher.get()) { _username = mo["username"].tostring(); } // remove domain part username string[] usernameparts = _...
Read more