Related
I have read a lot about the pain of replicate the easy robust option from STATA to R to use robust standard errors. I replicated following approaches: StackExchange and Economic Theory Blog. They work but the problem I face is, if I want to print my results using the stargazer function (this prints the .tex code for Latex files).
Here is the illustration to my problem:
reg1 <-lm(rev~id + source + listed + country , data=data2_rev)
stargazer(reg1)
This prints the R output as .tex code (non-robust SE) If i want to use robust SE, i can do it with the sandwich package as follow:
vcov <- vcovHC(reg1, "HC1")
if I now use stargazer(vcov) only the output of the vcovHC function is printed and not the regression output itself.
With the package lmtest() it is possible to print at least the estimator, but not the observations, R2, adj. R2, Residual, Residual St.Error and the F-Statistics.
lmtest::coeftest(reg1, vcov. = sandwich::vcovHC(reg1, type = 'HC1'))
This gives the following output:
t test of coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -2.54923 6.85521 -0.3719 0.710611
id 0.39634 0.12376 3.2026 0.001722 **
source 1.48164 4.20183 0.3526 0.724960
country -4.00398 4.00256 -1.0004 0.319041
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
How can I add or get an output with the following parameters as well?
Residual standard error: 17.43 on 127 degrees of freedom
Multiple R-squared: 0.09676, Adjusted R-squared: 0.07543
F-statistic: 4.535 on 3 and 127 DF, p-value: 0.00469
Did anybody face the same problem and can help me out?
How can I use robust standard errors in the lm function and apply the stargazer function?
You already calculated robust standard errors, and there's an easy way to include it in the stargazeroutput:
library("sandwich")
library("plm")
library("stargazer")
data("Produc", package = "plm")
# Regression
model <- plm(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp,
data = Produc,
index = c("state","year"),
method="pooling")
# Adjust standard errors
cov1 <- vcovHC(model, type = "HC1")
robust_se <- sqrt(diag(cov1))
# Stargazer output (with and without RSE)
stargazer(model, model, type = "text",
se = list(NULL, robust_se))
Solution found here: https://www.jakeruss.com/cheatsheets/stargazer/#robust-standard-errors-replicating-statas-robust-option
Update I'm not so much into F-Tests. People are discussing those issues, e.g. https://stats.stackexchange.com/questions/93787/f-test-formula-under-robust-standard-error
When you follow http://www3.grips.ac.jp/~yamanota/Lecture_Note_9_Heteroskedasticity
"A heteroskedasticity-robust t statistic can be obtained by dividing an OSL estimator by its robust standard error (for zero null hypotheses). The usual F-statistic, however, is invalid. Instead, we need to use the heteroskedasticity-robust Wald statistic."
and use a Wald statistic here?
This is a fairly simple solution using coeftest:
reg1 <-lm(rev~id + source + listed + country , data=data2_rev)
cl_robust <- coeftest(reg1, vcov = vcovCL, type = "HC1", cluster = ~
country)
se_robust <- cl_robust[, 2]
stargazer(reg1, reg1, cl_robust, se = list(NULL, se_robust, NULL))
Note that I only included cl_robust in the output as a verification that the results are identical.
I'm using GHC 8.4.2 on Windows. I have this program that depends on the library red-black-record, version 2.0.2.2:
{-# LANGUAGE DataKinds, TypeApplications #-}
module Main where
import Data.RBR (FromList,Delete,Variant,I,injectI,winnowI,match)
import GHC.TypeLits
type Phase01 = FromList '[
'("ctor1",Int), '("ctor2",Bool), '("ctor4",Char), '("ctor3",Char),
'("ctor6",Char), '("ctor5",Char), '("ctor10",Char), '("ctor11",Char),
'("ctor13",Char), '("ctor14",Char), '("ctor39",Char), '("ctor46",Char),
'("ctor47",Char), '("ctor44",Char), '("ctor43",Char), '("ctor7",Char),
'("ctor9",Char), '("ctor20",Char), '("ctor45",Char), '("ctor21",Char),
'("ctor48",Char), '("ctor49",Char), '("ctor50",Char), '("ctor41",Char),
'("ctor33",Char), '("ctor32",Char), '("ctor42",Char), '("ctor22",Char),
'("ctor23",Char), '("ctor8",Char), '("ctor40",Char), '("ctor29",Char),
'("ctor24",Char), '("ctor38",Char), '("ctor25",Char), '("ctor26",Char),
'("ctor27",Char), '("ctor28",Char), '("ctor36",Char), '("ctor52",Char),
'("ctor51",Char), '("ctor53",Char), '("ctor12",Char), '("ctor54",Char),
'("ctor15",Char), '("ctor31",Char), '("ctor30",Char), '("ctor34",Char),
'("ctor35",Char), '("ctor17",Char), '("ctor16",Char), '("ctor18",Char),
'("ctor19",Char), '("ctor37",Char)
]
type Phase02 = Delete "ctor1" Int Phase01
main :: IO ()
main = print (match #"ctor17" (fromPhase1ToPhase2 (injectI #"ctor1" 2)))
where
fromPhase1ToPhase2 :: Variant I Phase01 -> Variant I Phase02
fromPhase1ToPhase2 v = case winnowI #"ctor1" #Int v of
Right z -> injectI #"ctor2" False
Left l -> l
(The library itself is not important, except as an example of code that makes heavy use of type families.)
This code takes ~ 9 seconds to compile on my machine. But when I try to move the fromPhase1ToPhase2 function out of the where clause and make it a top-level function, compilation time spikes to ~ 29 seconds!
Is there a reason why lifting a function to the top level makes it compile slower?
Edit: as another data point, moving the function to the top-level but using -XPartialTypeSignatures like this:
{-# LANGUAGE PartialTypeSignatures #-}
{-# OPTIONS_GHC -Wno-partial-type-signatures #-}
fromPhase1ToPhase2 :: Variant I _ -> Variant I _
fromPhase1ToPhase2 v = case winnowI #"ctor1" #Int #Phase01 v of
Right z -> injectI #"ctor2" False
Left l -> l
keeps the original compilation time of ~ 9 seconds.
I tried for days to write a NLTK grammar to convert simple French sentences into logical formulas. My problem can be similar with English sentences. My goal is that this grammar accepts several orders (home automation) and converts them into logical formulas. Some examples of orders:
Turn on the light:
exists x.(turn_on(x) & light(x))
Turn on the green light:
exists x.(turn_on(x) & light(x) & green(x))
Turn on the light of the kitchen
exists x.(turn_on(x) & light(x) & exists y.(kitchen(y) & in(x, y)))
In these examples, the word turn_on is not really a logical predicate. It will be used in the next step of my program (when it will convert this formula into another representation).
However, I have many difficulties to write the rule about possession relationship. I would like that the rule accepts an "infinite" recursion like:
turn on the light of the kitchen (the light belongs to the kitchen in my database)
turn on the light of the kitchen of the house (the kitchen belongs to the house in my database)
turn on the light of the kitchen of the house of [...] (etc.)
I succeeded to convert the first sentence but not the others. Here my grammar (I translate French to English for a better understanding):
% start SV
SV[SEM=<?v(?sn)>] -> V[SEM=?v] SN[SEM=?sn]
SN[SEM=<?ap(?sn1, ?sn2)>] -> SN[SEM=?sn1] AP[SEM=?ap] SN[SEM=?sn2]
SN[SEM=<?ad(?n)>] -> AD[SEM=?ad] N[SEM=?n]
SN[SEM=?n] -> N[SEM=?n]
N[SEM=<?adj(?n)>] -> ADJ[SEM=?adj] N[SEM=?n]
V[SEM=<\P.P(\x.turn_on(x))>] -> 'turn' 'on'
N[SEM=<\x.light(x)>] -> 'light'
N[SEM=<\x.kitchen(x)>] -> 'kitchen'
N[SEM=<\x.house(x)>] -> 'house'
ADJ[SEM=<\P x.(P(x) & big(x))>] -> 'big'
ADJ[SEM=<\P x.(P(x) & green(x))>] -> 'green'
AD[SEM=<\P Q.exists x.(P(x) & Q(x))>] -> 'the'
AP[SEM=<\P Q R.Q(\x.P(\y.(in(y,x) & R(y))))>] -> 'of'
With this grammar and the order "turn on the light of the kitchen", I get:
exists x.(kitchen(x) & exists z1.(light(z1) & in(z1,x) & turn_on(z1)))
But, for the order "turn on the light of the kitchen of the house":
exists x.(house(x) & exists z5.(kitchen(z5) & exists z2.(light(z2) & in(z2,z5) & in(z2,x) & turn_on(z2))))
To be more readable, the same formula without the "exists":
(house(x4) & kitchen(x6) & light(x7) & in(x7,x6) & in(x7,x4) & turn_on(x7))
There is a problem with the "in" predicates. Indeed, I want that the light is in the kitchen and that the kitchen is in the house. However, in this case, the light is in the kitchen and in the house (yes, it's true, but I don't want that =/). Here's what I would like:
(house(x4) & kitchen(x6) & light(x7) & in(x7,x6) & in(x6,x4) & turn_on(x7))
the difference -----^
I tried several methods but none of them worked... Can you help me please? I don't know if it's possible with my grammar. My knowledge on logic and lambda calcul are limited, I only just beginning to get interested in these topics.
EDIT:
Here is the python code that I use for my tests:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import nltk
def exec(parser, query):
try:
trees = list(parser.parse(query.split()))
except ValueError:
print('Invalid query')
return
if len(trees) == 0:
print('Invalid query')
return
print('query: %s' % query)
print('results:')
for t in trees:
sem = t.label()['SEM']
print('\t%s' % sem)
print('')
if __name__ == '__main__':
parser = nltk.load_parser('./en_grammar.fcfg')
exec(parser, 'turn on the light')
exec(parser, 'turn on the light of the kitchen')
exec(parser, 'turn on the light of the kitchen of the house')
Thanks a lot and sorry for my English.
It is hard to say that an existential quantifier is the logical form of an imperative sentence. However, your question lies in another problem.
It seems that you have an ambiguous grammar. Specially when you intrepret the x of y with in(x, y) function, it is imaginable to have similar ambiguity as in second phrase:
the light of the kitchen in the house .
the ball of the kid in the yard .
The ball which is in the yard.
The kid who is the yard.
Your grammar based on your code produces these two interpretations for desired sentence:
query: turn on the light of the kitchen of the house
results:
exists x.(house(x) & exists z5.(kitchen(z5) & exists z2.(light(z2) & in(z2,z5) & in(z2,x) & turn_on(z2))))
exists x.(house(x) & exists z3.(kitchen(z3) & in(z3,x) & exists z6.(light(z6) & in(z6,z3) & turn_on(z6))))
In second interpretation: house(x) & exists z3.(kitchen(z3) & in(z3,x) ... is exact thing that you want.
UPDATE:
Let's try to avoid the ambiguity in chains of x of y of z.
One very fast solution to force x of (y of z) instead of (x of y) of z is to track the of usage in all noun phrases, and then force it to have no OF on the left side of the of:
SN[SEM=<?ap(?sn1, ?sn2)>, +OF] -> SN[SEM=?sn1, -OF] AP[SEM=?ap] SN[SEM=?sn2]
SN[SEM=<?ad(?n)>, -OF] -> AD[SEM=?ad] N[SEM=?n]
SN[SEM=?n, -OF] -> N[SEM=?n]
I'm a begginer at Prolog and I need some help with this exercise, this is the knowledge base provided:
album(‘R. Stevie Moore’, ‘Manuscription’).
album(‘Lane Steinberg’, ‘Manuscription’).
album(‘R. Stevie Moore’, ‘The Yung & Moore Show’).
album(‘Yukio Yung’, ‘The Yung & Moore Show’).
album(‘Jessie Evans’, ‘Autonervous’).
album(‘Bettina Koster’, ‘Autonervous’).
album(‘Lucia Pamela’, ‘Walkin on the moon’).
album(‘Shooby Taylor’, ‘The Human Horn’).
album(‘Tiny Tim’, ‘God Bless Tiny Tim’).
album(‘The Legendary Stardust Cowboy’, ‘Rock-It to Stardom’).
vinil(‘Rock-It to Stardom’).
vinil(‘Walking on the Moon’).
cd( ‘God Bless Tiny Tim’).
cd(‘Walking on the Moon’).
cd(‘Autonervous’).
cd(‘Manuscription’).
cassette(‘The Human Horn’).
cassette(‘The Yung & Moore Show’).
mp3(‘Walkin on the Moon’).
I need to make a query that will return to me all the albums that were made by only one musician.
Thanks for your help :)
In the comments you have provided your code album(X,B),album(Y,C), B \= C. Actually, this is not too far away from a correct solution.
A correct solution could be:
one_musician_album(X) :-
album(A, X),
\+ (album(B, X), A \= B).
The meaning of the predicate is: an album X is a "one-musician album" if this album is authored by some musician A and it's not possible to find a different musician B authored the album X.
Test run:
?- one_musician_album(X).
X = 'Walkin on the moon' ;
X = 'The Human Horn' ;
X = 'God Bless Tiny Tim' ;
X = 'Rock-It to Stardom'.
To get all answers you have to type ';' after each answer.
Maybe this is not needed for you, but it's possible to get all answers in a list with findall:
?- findall(X, one_musician_album(X), Albums).
Albums = ['Walkin on the moon', 'The Human Horn', 'God Bless Tiny Tim', 'Rock-It to Stardom'].
Here is a generalization, based on 'all solutions' builtin bagof/3.
Note that
?- bagof(A, album(A,T), As).
T = 'Autonervous',
As = ['Jessie Evans', 'Bettina Koster'] ;
T = 'God Bless Tiny Tim',
As = ['Tiny Tim']
....
then, restricting the authors (As) to be a list of a single element, we get albums with a single author
?- bagof(A, album(A,T), [A]).
A = 'Tiny Tim',
T = 'God Bless Tiny Tim' ;
...
then, we could use findall/3, as in the good answer by Sergey, or, again, bagof/3, with explicit quantification:
?- bagof(T, A^bagof(A, album(A, T), [A]), Ts).
Ts = ['God Bless Tiny Tim', 'Rock-It to Stardom', 'The Human Horn', 'Walkin on the moon'].
I am new to Haskell. Previously I have programmed in Python and Java. When I am debugging some code I have a habit of littering it with print statements in the middle of code. However doing so in Haskell will change semantics, and I will have to change my function signatures to those with IO stuff. How do Haskellers deal with this? I might be missing something obvious. Please enlighten.
Other answers link the official doco and the Haskell wiki but if you've made it to this answer let's assume you bounced off those for whatever reason. The wikibook also has an example using Fibonacci which I found more accessible. This is a deliberately basic example which might hopefully help.
Let's say we start with this very simple function, which for important business reasons, adds "bob" to a string, then reverses it.
bobreverse x = reverse ("bob" ++ x)
Output in GHCI:
> bobreverse "jill"
"llijbob"
We don't see how this could possibly be going wrong, but something near it is, so we add debug.
import Debug.Trace
bobreverse x = trace ("DEBUG: bobreverse" ++ show x) (reverse ("bob" ++ x))
Output:
> bobreverse "jill"
"DEBUG: bobreverse "jill"
llijbob"
We are using show just to ensure x is converted to a string correctly before output. We also added some parenthesis to make sure the arguments were grouped correctly.
In summary, the trace function is a decorator which prints the first argument and returns the second. It looks like a pure function, so you don't need to bring IO or other signatures into the functions to use it. It does this by cheating, which is explained further in the linked documentation above, if you are curious.
Read this. You can use Debug.Trace.trace in place of print statements.
I was able to create a dual personality IO / ST monad typeclass, which will print debug statements when a monadic computation is typed as IO, them when it's typed as ST. Demonstration and code here: Haskell -- dual personality IO / ST monad? .
Of course Debug.Trace is more of a swiss army knife, especially when wrapped with a useful special case,
trace2 :: Show a => [Char] -> a -> a
trace2 name x = trace (name ++ ": " ++ show x) x
which can be used like (trace2 "first arg" 3) + 4
edit
You can make this even fancier if you want source locations
{-# LANGUAGE TemplateHaskell #-}
import Language.Haskell.TH
import Language.Haskell.TH.Syntax as TH
import Debug.Trace
withLocation :: Q Exp -> Q Exp
withLocation f = do
let error = locationString =<< location
appE f error
where
locationString :: Loc -> Q Exp
locationString loc = do
litE $ stringL $ formatLoc loc
formatLoc :: Loc -> String
formatLoc loc = let file = loc_filename loc
(line, col) = loc_start loc
in concat [file, ":", show line, ":", show col]
trace3' (loc :: String) msg x =
trace2 ('[' : loc ++ "] " ++ msg) x
trace3 = withLocation [| trace3' |]
then, in a separate file [from the definition above], you can write
{-# LANGUAGE TemplateHaskell #-}
tr3 x = $trace3 "hello" x
and test it out
> tr3 4
[MyFile.hs:2:9] hello: 4
You can use Debug.Trace for that.
I really liked Dons short blog about it:
https://donsbot.wordpress.com/2007/11/14/no-more-exceptions-debugging-haskell-code-with-ghci/
In short: use ghci, example with a program with code called HsColour.hs
$ ghci HsColour.hs
*Main> :set -fbreak-on-exception
*Main> :set args "source.hs"
Now run your program with tracing on, and GHCi will stop your program at the call to error:
*Main> :trace main
Stopped at (exception thrown)
Ok, good. We had an exception… Let’s just back up a bit and see where we are. Watch now as we travel backwards in time through our program, using the (bizarre, I know) “:back” command:
[(exception thrown)] *Main> :back
Logged breakpoint at Language/Haskell/HsColour/Classify.hs:(19,0)-(31,46)
_result :: [String]
This tells us that immediately before hitting error, we were in the file Language/Haskell/HsColour/Classify.hs, at line 19. We’re in pretty good shape now. Let’s see where exactly:
[-1: Language/Haskell/HsColour/Classify.hs:(19,0)-(31,46)] *Main> :list
18 chunk :: String -> [String]
vv
19 chunk [] = head []
20 chunk ('\r':s) = chunk s -- get rid of DOS newline stuff
21 chunk ('\n':s) = "\n": chunk s
^^