hadoop - Reducer is not being called -



hadoop - Reducer is not being called -

this code ebola info set. here reducer not beingness called @ all. mapper output beingness printed.

the driver class:

import org.apache.hadoop.conf.configuration; import org.apache.hadoop.fs.path; import org.apache.hadoop.io.text; import org.apache.hadoop.mapreduce.*; import org.apache.hadoop.mapreduce.lib.input.fileinputformat; import org.apache.hadoop.mapreduce.lib.input.keyvaluetextinputformat; import org.apache.hadoop.mapreduce.lib.output.fileoutputformat; import org.apache.hadoop.mapreduce.lib.output.textoutputformat; public class ebola { public static void main(string[] args) throws exception , arrayindexoutofboundsexception{ configuration con1 = new configuration(); con1.set("mapreduce.input.keyvaluelinerecordreader.key.value.separator", " "); job job1 = new job(con1, "ebola"); job1.setjarbyclass(ebola.class); job1.setinputformatclass(keyvaluetextinputformat.class); job1.setoutputformatclass(textoutputformat.class); job1.setoutputkeyclass(text.class); job1.setoutputvalueclass(text.class); job1.setmapperclass(ebolamapper.class); job1.setreducerclass(ebolreducer.class); fileinputformat.addinputpath(job1, new path(args[0])); fileoutputformat.setoutputpath(job1, new path(args[1])); job1.waitforcompletion(true); } }

this mapper:

import java.io.ioexception; import org.apache.hadoop.io.*; import org.apache.hadoop.mapreduce.mapper; public class ebolamapper extends mapper <text, text, text, text> { public void map(text key, text value, context con) throws ioexception, interruptedexception { text cumvalues = new text(); string record = value.tostring(); string p[] = record.split(" ",2); string cases = p[0]; string death = p[1]; string cvalues = death + "->" + cases; cumvalues.set(cvalues); con.write(key, cumvalues); } }

finally, reducer:

import java.io.ioexception; import org.apache.hadoop.io.text; import org.apache.hadoop.mapreduce.reducer; public class ebolreducer extends reducer<text, text, text, text> { public void reduce(text key, text value, context con) throws ioexception{ text cumulvalues = new text(); string cumval = value.tostring(); string[] p = cumval.split("->",2); string death = p[0]; string cases = p[1]; float d = float.parsefloat(death); float c = float.parsefloat(cases); float perc = (d/c)*100; string percent = string.valueof(perc); cumulvalues.set(percent); con.write(key,cumulvalues); } }

the output mapper output. reducer not beingness called. help appreciated.

instead of public void reduce(text key, text value, context con)

you need utilize iterable .

public void reduce(text key, iterable< text> value, context con)

hadoop reducers

Comments

Post a Comment

Popular posts from this blog

xslt - DocBook 5 to PDF transform failing with error: "fo:flow" is missing child elements. Required content model: marker* -

xslt - DocBook 5 to PDF transform failing with error: "fo:flow" is missing child elements. Required content model: marker* - i've inherited doc publishing process takes docbook , builds html , pdf output, using apache fop both. project had been started, never completed. lots of tweaking, i've able of doc sets build (20+), except handful, , fail severe code 1 of these 2 conditions: "fo:flow" missing kid elements. required content model: marker* "fo:block" not valid kid of "fo:root" the xml output comes out of docbook , docbook-fop checks out well-formed. the fop error in 1 book, "endecagloss", example, points end of string: <fo:flow flow-name="xsl-region-body" start-indent="4pc" end-indent="0pt"/> here’s log output in case: [java] severe: javax.xml.transform.transformerexception: file:/scratch/publishing/hudson/jobs/build-main-endeca-documentation/workspace/serverdoc/w...
Read more

mediawiki - How do I insert tables inside infoboxes on Wikia pages? -

mediawiki - How do I insert tables inside infoboxes on Wikia pages? - i want insert tables in infoboxes. example, have proteinbox i've developed using infobox template (http://health-and-medicine.wikia.com/wiki/template:proteinbox), , fasta field big fit comfortably in proteinbox. hence collapsible table within box. this tried: {{infobox | row 1 = {| class = "mw-collapsible mw-collapsed wikitable" |- | <!-- table content --> |} }} your problem is, putting nested table within template. pipe characters ( | ) in table syntax collides pipe usage in templates. the commonly used hack around create template called template:! , containing pipe character, , utilize when need set tables, parser functions or other stuff using pipe characters, within templates. table (with every | replaced {{!}} ): {{{!}} {{!}}- {{!}} a1 {{!}} b1 {{!}}- {{!}} a2 {{!}} b2 {{!}}} ...the equivalent of {| |- | a1 | b1 |- | a2 | b2 |} furthermore, hav...
Read more

Local Service User Logged into Windows -

Local Service User Logged into Windows - i created windows service tracks app server user connected through load balancer. it retrieves user logged windows on machine, cross references active directory (ad) business relationship homecoming first name, lastly name , e-mail (for notification through website created project) there 1 scheme on network users remote system, vista system, reason returning values "local service" logged machine, it's weird , security guys impossible "local service" business relationship logged windows. here how user logged windows managementscope ms = new managementscope("\\\\.\\root\\cimv2"); objectquery query = new objectquery("select * win32_computersystem"); managementobjectsearcher searcher = new managementobjectsearcher(ms, query); foreach (managementobject mo in searcher.get()) { _username = mo["username"].tostring(); } // remove domain part username string[] usernameparts = _...
Read more