R Logistic regression on ffdf objects -



R Logistic regression on ffdf objects -

i have built logistic regression model using glm function stats package. predict outcome of model on big number of values, stored in "ffdf" object (see ff package), not find how proceed:

how can create subset of ffdf object, in order maintain variables (i.e. columns) used in prediction? - needed specify input in predict function

how should proceed next? function should used between predict(), predict.glm(), predict.bigglm() (maybe biglm bundle helpful)?

thank in advance views on this!

best regards

update

thank feedback bondeddust. allow me more precise, indeed coding question, aiming @ performing logistic regression based on ffdf object (learning dataset), , predict outcome of model ffdf object (test dataset).

(1/3) learning info set: ffdf object (created ff package).

` class(train.random.sample)` > [1] "ffdf"

below construction of ffdf object in case of needs:

`str(train.random.sample) ` > list of 3 $ virtual: 'data.frame': 27 obs. of 7 variables: .. $ virtualvmode : chr "integer" "integer" "integer" "integer" ... .. $ asis : logi false false false false false false ... .. $ virtualismatrix : logi false false false false false false ... .. $ physicalismatrix : logi false false false false false false ... .. $ physicalelementno: int 1 2 3 4 5 6 7 8 9 10 ... .. $ physicalfirstcol : int 1 1 1 1 1 1 1 1 1 1 ... .. $ physicallastcol : int 1 1 1 1 1 1 1 1 1 1 ... .. - attr(*, "dim")= int 500000 27 .. - attr(*, "dimorder")= int 1 2 $ physical: list of 27 .. $ id : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ click : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ hr : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ c1 : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ banner_pos : list() .. ..- attr(*, "physical")=class 'ff_pointer' <externalptr> .. .. ..- attr(*, "vmode")= chr "integer" .. .. ..- attr(*, "maxlength")= int 500000 .. .. ..- attr(*, "pattern")= chr "ffdf" .. .. ..- attr(*, "filename")= chr "anonymized.ff" .. .. ..- attr(*, "pagesize")= int 65536 .. .. ..- attr(*, "finalizer")= chr "delete" .. .. ..- attr(*, "finonexit")= logi true .. .. ..- attr(*, "readonly")= logi false .. .. ..- attr(*, "caching")= chr "mmnoflush" .. ..- attr(*, "virtual")= list() .. .. ..- attr(*, "length")= int 500000 .. .. ..- attr(*, "symmetric")= logi false .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ site_id : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ site_domain : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ site_category : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ app_id : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ app_domain : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ app_category : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ device_id : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ device_ip : list() …. .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ device_os : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ device_make : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ device_model : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ device_type : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ device_conn_type : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ device_geo_country: list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" .. $ c17 : list() … .. .. - attr(*, "class") = chr [1:2] "ff_vector" "ff" $ row.names: null - attributes: list of 2 .. $ names: chr [1:3] "virtual" "physical" "row.names" .. $ class: chr "ffdf"

(2/3) logistic regression based on learning dataset:

objective learn/ predict ‘click’ outcome based on ‘baser_pos’ input

`logreg1 <- glm(click ~ banner_pos, info = train.random.sample, family = "binomial") summary(logreg1)` > call: glm(formula = click ~ banner_pos, family = "binomial", info = train.random.sample) deviance residuals: min 1q median 3q max -1.0555 -0.6495 -0.5951 -0.5951 1.9071 coefficients: estimate std. error z value pr(>|z|) (intercept) -1.641416 0.004702 -349.12 <2e-16 xxx banner_pos 0.192534 0.007595 25.35 <2e-16 xxx --- signif. codes: 0 ‘xxx’ 0.001 ‘xx’ 0.01 ‘x’ 0.05 ‘.’ 0.1 ‘ ’ 1 (dispersion parameter binomial family taken 1) null deviance: 458848 on 499999 degrees of freedom residual deviance: 458215 on 499998 degrees of freedom aic: 458219 number of fisher scoring iterations: 4 `class(logreg1)`> [1] "glm" "lm"

(3/3)test dataset: ffdf object (created ff package).

`class(df.test)` > [1] "ffdf"

test dataset construction identical training dataset, ~4.8m rows

`str(df.test)`> list of 3 $ virtual: 'data.frame': 26 obs. of 7 variables: .. $ virtualvmode : chr "integer" "integer" "integer" "integer" ... .. $ asis : logi false false false false false false ... .. $ virtualismatrix : logi false false false false false false ... .. $ physicalismatrix : logi false false false false false false ... .. $ physicalelementno: int 1 2 3 4 5 6 7 8 9 10 ... .. $ physicalfirstcol : int 1 1 1 1 1 1 1 1 1 1 ... .. $ physicallastcol : int 1 1 1 1 1 1 1 1 1 1 ... .. - attr(*, "dim")= int 4769401 26 .. - attr(*, "dimorder")= int 1 2 $ physical: list of 26 …

i not succeed in predicting click outcome. first tried create dataframe or ffdf object containing banner_pos variable:

`modeldata <- df.test[["banner_pos"]]`

then tried predict outcome:

`predict.glm(object = logreg1, newdata = modeldata, type = "response")` error in as.data.frame.default(data) : cannot coerce class "c("ff_vector", "ff")" data.frame

is there wrong in code? should utilize other function leveraging other packages such biglm? many in advance views on issue, best regards

something similar score ffdf alongside glm.

require(ff) df.test$score <- ff(as.numeric(na), length = nrow(df.test)) chunks <- chunk(df.test) for(chunkrangeindex in chunks){ df.test$score[chunkrangeindex] <- predict(object = logreg1, newdata = df.test[chunkrangeindex, ], type = "response") }

r regression ff

Comments

Post a Comment

Popular posts from this blog

xslt - DocBook 5 to PDF transform failing with error: "fo:flow" is missing child elements. Required content model: marker* -

xslt - DocBook 5 to PDF transform failing with error: "fo:flow" is missing child elements. Required content model: marker* - i've inherited doc publishing process takes docbook , builds html , pdf output, using apache fop both. project had been started, never completed. lots of tweaking, i've able of doc sets build (20+), except handful, , fail severe code 1 of these 2 conditions: "fo:flow" missing kid elements. required content model: marker* "fo:block" not valid kid of "fo:root" the xml output comes out of docbook , docbook-fop checks out well-formed. the fop error in 1 book, "endecagloss", example, points end of string: <fo:flow flow-name="xsl-region-body" start-indent="4pc" end-indent="0pt"/> here’s log output in case: [java] severe: javax.xml.transform.transformerexception: file:/scratch/publishing/hudson/jobs/build-main-endeca-documentation/workspace/serverdoc/w...
Read more

mediawiki - How do I insert tables inside infoboxes on Wikia pages? -

mediawiki - How do I insert tables inside infoboxes on Wikia pages? - i want insert tables in infoboxes. example, have proteinbox i've developed using infobox template (http://health-and-medicine.wikia.com/wiki/template:proteinbox), , fasta field big fit comfortably in proteinbox. hence collapsible table within box. this tried: {{infobox | row 1 = {| class = "mw-collapsible mw-collapsed wikitable" |- | <!-- table content --> |} }} your problem is, putting nested table within template. pipe characters ( | ) in table syntax collides pipe usage in templates. the commonly used hack around create template called template:! , containing pipe character, , utilize when need set tables, parser functions or other stuff using pipe characters, within templates. table (with every | replaced {{!}} ): {{{!}} {{!}}- {{!}} a1 {{!}} b1 {{!}}- {{!}} a2 {{!}} b2 {{!}}} ...the equivalent of {| |- | a1 | b1 |- | a2 | b2 |} furthermore, hav...
Read more

Local Service User Logged into Windows -

Local Service User Logged into Windows - i created windows service tracks app server user connected through load balancer. it retrieves user logged windows on machine, cross references active directory (ad) business relationship homecoming first name, lastly name , e-mail (for notification through website created project) there 1 scheme on network users remote system, vista system, reason returning values "local service" logged machine, it's weird , security guys impossible "local service" business relationship logged windows. here how user logged windows managementscope ms = new managementscope("\\\\.\\root\\cimv2"); objectquery query = new objectquery("select * win32_computersystem"); managementobjectsearcher searcher = new managementobjectsearcher(ms, query); foreach (managementobject mo in searcher.get()) { _username = mo["username"].tostring(); } // remove domain part username string[] usernameparts = _...
Read more