'invalid memory address' error with go-mssql - go

I'm having an issue that I can't seem to resolve, probably due to my inexperience with GO. I have the following code working on one server, but not on another. Here is the code:
// Build out the connection string to the database, and then open the connection to the database.
connString := fmt.Sprintf("server=%s;user id=%s;password=%s;port=%d", *server, *user, *password, *port)
if *debug { fmt.Printf(" connString:%s\n", connString) }
db, err = sql.Open("mssql", connString)
if err != nil { log.Fatal("Open connection failed:", err.Error()) }
err = db.Ping()
if err != nil {
fmt.Println("Cannot connect: ", err.Error())
return
}
rows, _ := db.Query( "SELECT Zip FROM Zip_Rural WHERE Zip = ?", ZipCode[0:5] )
defer rows.Close()
if !rows.Next() {
acreageRequirement = .5
}
On the line that reads if !rows.Next() I get the following error:
panic: runtime error: invalid memory address or nil pointer dereference
panic: runtime error: invalid memory address or nil pointer dereference [signal 0xc0000005 code=0x0 addr=0x20 pc=0x477918]
This same code works just fine on another server, running GO version 1.4.2 on both machines. I have a feeling I just have some bad syntax somewhere in here, but have no idea what the real problem is. A call to db.Exec within the same file works just fine, which leads me to believe that my database connection is perfectly fine, but for some reason db.Query is not executing correctly.

There is probably an error with db.Query. Check your error and if it is not nil, assume that rows is nil. i.e. calling rows.Next() will segfault.
If you display the error, you will probably find out what the issue is.

Related

Apache Tinkerpop gremlin-go driver Next() on AddE returns 'E0903: there are no results left'

Potential bug in gremlin-go when creating new edge and returning next result. Before creating a JIRA bug ticket for the following, I just want to make sure, it is not an user error/ my fault.
Version of Apache Tinkerpop gremlin-go used to connect and interact with JanusGraph (0.6.2): github.com/apache/tinkerpop/gremlin-go/v3 v3.6.1.
Example without transaction:
g := gremlingo.Traversal_().WithRemote(gremlingo.NewDriverRemoteConnection(source))
rv, err := g.AddV("v").Next()
fmt.Println(err)
fmt.Println(rv)
v, err := rv.GetVertex()
fmt.Println(err)
fmt.Println(v)
fmt.Println(v.Id)
re, err := g.AddE("e").Property("some_property", "some-property").From(__.V(v.Id)).To(__.V(v.Id)).Next()
fmt.Println(err)
fmt.Println(re)
Returns:
<nil>
result{object=v[%!s(int64=24776)] class=*gremlingo.Vertex}
<nil>
v[%!s(int64=24776)]
24776
E0903: there are no results left
<nil>
Example with transaction:
g := gremlingo.Traversal_().WithRemote(gremlingo.NewDriverRemoteConnection(source))
tx := g.Tx()
gtx, err := tx.Begin()
if err != nil {
tx.Close()
}
rv, err = gtx.AddV("v").Next()
if err != nil {
fmt.Println(err)
tx.Rollback()
}
fmt.Println(rv)
v, err = rv.GetVertex()
if err != nil {
fmt.Println(err)
tx.Rollback()
}
fmt.Println(v)
fmt.Println(v.Id)
re, err = gtx.AddE("e").Property("some_property", "some-property").From(__.V(v.Id)).To(__.V(v.Id)).Next()
if err != nil {
fmt.Println(err)
tx.Rollback()
}
fmt.Println(re)
err = txx.Commit()
if err != nil {
tx.Rollback()
}
Returns:
Creating Session based connection
Connecting.
result{object=v[%!s(int64=28872)] class=*gremlingo.Vertex}
v[%!s(int64=28872)]
28872
panic: runtime error: slice bounds out of range [:7623] with capacity 512
goroutine 133 [running]:
github.com/apache/tinkerpop/gremlin-go/v3/driver.readString(0xc00000ec60, 0xc0003b2fa0)
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/graphBinary.go:912 +0x145
github.com/apache/tinkerpop/gremlin-go/v3/driver.readUnqualified(0xc00000ec60, 0xc0003b2fa0, 0x3, 0x0?)
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/graphBinary.go:1308 +0x362
github.com/apache/tinkerpop/gremlin-go/v3/driver.edgeReader(0xc4e860?, 0xc0003b2fa0)
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/graphBinary.go:1079 +0x7d
github.com/apache/tinkerpop/gremlin-go/v3/driver.readFullyQualifiedNullable(0xc00000ec60, 0xc0003b2fa0, 0xe0?)
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/graphBinary.go:1327 +0x3f1
github.com/apache/tinkerpop/gremlin-go/v3/driver.traverserReader(0xc00000ec60, 0xc0003b2fa0)
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/graphBinary.go:1159 +0xdc
github.com/apache/tinkerpop/gremlin-go/v3/driver.readFullyQualifiedNullable(0xc00000ec60, 0xc0003b2fa0, 0x0?)
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/graphBinary.go:1327 +0x3f1
github.com/apache/tinkerpop/gremlin-go/v3/driver.readList(0xc00000ec60, 0xc0003b2fa0)
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/graphBinary.go:953 +0x133
github.com/apache/tinkerpop/gremlin-go/v3/driver.readFullyQualifiedNullable(0xc00000ec60, 0xc0003b2fa0, 0xc0?)
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/graphBinary.go:1327 +0x3f1
github.com/apache/tinkerpop/gremlin-go/v3/driver.graphBinarySerializer.deserializeMessage({0xc00051ec00?}, {0xc0005b2200, 0xc5, 0x200})
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/serializer.go:214 +0x3fa
github.com/apache/tinkerpop/gremlin-go/v3/driver.(*gremlinServerWSProtocol).readLoop(0xc00051ec00, 0x5456ea?, 0x1c?)
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/protocol.go:75 +0x1e2
created by github.com/apache/tinkerpop/gremlin-go/v3/driver.newGremlinServerWSProtocol
/go/pkg/mod/github.com/apache/tinkerpop/gremlin-go/v3#v3.6.1/driver/protocol.go:200 +0x250
When adding .ElementMap():
gtx.AddE("e").Property("some_property", "some-property").From(__.V(v.Id)).To(__.V(v.Id)).ElementMap().Next()
Returns:
Creating Session based connection
Connecting.
result{object=v[%!s(int64=16576)] class=*gremlingo.Vertex}
v[%!s(int64=16576)]
16576
2023/01/10 11:49:46 Error occurred during operation gremlinServerWSProtocol.readLoop(): 'E0408: unknown data type to deserialize 0x0'
2023/01/10 11:49:46 Read loop error 'E0408: unknown data type to deserialize 0x0', closing read loop.
2023/01/10 11:49:46 Connection error callback invoked, closing protocol.
2023/01/10 11:49:46 Connecting.
E0903: there are no results left
2023/01/10 11:49:47 Closing DriverRemoteConnection with url 'ws://janusgraph:8182/gremlin' with session '3071a486-6649-4f05-8baa-9603163936a3'
2023/01/10 11:49:47 Closing Client with url 'ws://janusgraph:8182/gremlin'
2023/01/10 11:49:47 Closing the connection.
Expected result via gremlin-console:
gremlin> g.addE("e").property("some_property", "some-property").from(__.V(24776)).to(__.V(24776)).next()
==>e[8bd-j48-q39-j48][24776-e->24776]
gremlin> g.addE("e").property("some_property", "some-property").from(__.V(24776)).to(__.V(24776)).elementMap().next()
==>id=8pl-j48-q39-j48
==>label=e
==>IN={id=24776, label=v}
==>OUT={id=24776, label=v}
==>some_property=some-property
Adding an Edge returns the Edge, and JanusGraph uses a special type RelationIdentifier as the Edge Id, which the Gremlin Go driver does not support. This causes the E0408: unknown data type to deserialize 0x0.
If you just want to create an Edge in the graph, a workaround is to not return it in the driver with g.addE().from([...]).to([...]).none().
More details in https://issues.apache.org/jira/browse/TINKERPOP-2828.

Go converting data to parquet

I am getting data from dynamodb and converting data to parquet files.
For the conversion to parquet files, I am using https://github.com/xitongsys/parquet-go library. But for some reason I am getting runtime error: invalid memory address or nil pointer dereference
I know this error means that a pointer is nil and I am trying to use it or dereference it.
Not sure why pw.WriteStop() is giving memory invalid error
code:
fw, err := local.NewLocalFileWriter(parquetFile)
if err != nil {
log.Errorf("local.NewLocalFileWriter() error - %s", err)
return err
}
pw, err := writer.NewParquetWriter(fw, new(Struct), int64(len(tenantList)))
if err != nil {
log.Errorf("writer.NewParquetWriter() error - %s", err)
return err
}
pw.RowGroupSize = 128 * 1024 * 1024 //128M
pw.CompressionType = parquet.CompressionCodec_SNAPPY
for _, data := range tenantList {
if err = pw.Write(data); err != nil {
return err
}
}
// this line gives memory invalid error
if err = pw.WriteStop(); err != nil {
return err
}
fw.Close()
The only error that I am getting is "runtime error: invalid memory address or nil pointer dereference". By printing statements in the "if" statements I found that invalid memory address is coming from "pw.WriteStop()"
Attaching a screenshot of the error; just to show that I am really getting only "runtime error: invalid memory address or nil pointer dereference"
I faced this too. Figured out that it is because I haven't specified 'type=BOOLEAN' for bool fields in the struct tags, e.g. parquet:"name=MyFieldName, type=BOOLEAN". For me, supplying BOOLEAN for bool, INT64 for int64 and UTF8 for string Go types worked.
Ideally the library should pick up data types automatically, but probably it's a feature not implemented, or it involves too much overhead and that's why intentionally not implemented.

Random REFUSED_STREAM making a POST multipart form request

This is the code to upload an apk file (several MB) to appetize.io using golang:
func uploadToAppetize(file multipart.File, branchName string, displayName string) (result *AppetizeRes, ok bool) {
file.Seek(0, 0)
url, _ := getUrl()
var buffer bytes.Buffer
writer := multipart.NewWriter(&buffer)
fileName := displayName + "/" + branchName
part, err := writer.CreateFormFile("file", fileName)
if err != nil {
fmt.Fprintf(os.Stderr, "Error creating form file %v: %v\n", fileName, err)
return nil, false
}
size, err := io.Copy(part, file)
if err != nil {
fmt.Fprintf(os.Stderr, "Error copying apk file data: %v\n", err)
return nil, false
}
fmt.Fprintf(os.Stdout, "Copied %v bytes for uploading to appetize...\n", size)
writer.Close()
response, err := http.Post(url, writer.FormDataContentType(), &buffer) // Random error on this line
if err != nil || response == nil {
fmt.Fprintf(os.Stderr, "Error occurred uploading apk data to appetize.io: %v %v\n", err, response)
return nil, false
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return nil, false
}
var appetizeRes AppetizeRes
if err := json.NewDecoder(response.Body).Decode(&appetizeRes); err != nil {
return nil, false
}
return &appetizeRes, true
}
However I am receiving an random error on the line http.Post(...). It returns a nil response and an error. The error is "stream error: stream ID 1; REFUSED_STREAM". It happens randomly but will surely happen the first time the go program make the request after launching.
This is the go version:
go version go1.8.1 darwin/amd64
This is the response header from the server if it doesn't fail:
I also run this program on another mac running go 1.6.*, I didn't remember I ever run into this issue on that mac.
Any idea what's going on?
There is a bug in the golang standard library net/http that fails to handle the REFUSED_STREAM http/2 error properly. This is what is likely going on:
golang client opens a TCP connection to the HTTP/2 www server, sets the maximum number of HTTP streams in the connection to 1000 and starts uploading immediately.
The HTTP/2 www server tells the golang client to only use a given number of streams, but the golang client has already started more than that amount of streams.
The HTTP/2 www server reacts to this by resetting the excess streams.
The lack of proper support for a retry in the golang net/http code causes the stream to fail after being reset by the server, and eventually the upload fails as well.
There is a ticket open about this issue on github: x/net/http2: retry requests rejected with REFUSED_STREAM - golang/go/issues/20985
**This issue has been closed in the master branch, the fix will be included in future Golang version 1.10

Unable to create session connecting to Cassandra

I am using this gocql package.
I am trying to get this example working.
func main() {
// connect to the cluster
cluster := gocql.NewCluster("192.168.1.1", "192.168.1.2", "192.168.1.3")
cluster.ProtoVersion = 3
cluster.Keyspace = "example"
cluster.Consistency = gocql.Quorum
session, err := cluster.CreateSession()
defer session.Close()
if err != nil {
fmt.Printf("%v\n", err)
return
}
uuid := gocql.TimeUUID()
fmt.Printf("UUID : %v", uuid)
query := session.Query(`INSERT INTO tweet (timeline, id, text) VALUES
(?, ?, ?)`, "me", uuid, "hello world")
fmt.Println("About to exec")
err = query.Exec()
// insert a tweet
if err != nil {
log.Fatal(err)
}
var id gocql.UUID
var text string
/* Search for a specific set of records whose 'timeline' column matches
* the value 'me'. The secondary index that we created earlier will be
* used for optimizing the search */
if err := session.Query(`SELECT id, text FROM tweet WHERE timeline = ?
LIMIT 1`,"me").Consistency(gocql.One).Scan(&id, &text); err != nil {
log.Fatal(err)
}
fmt.Println("Tweet:", id, text)
// list all tweets
iter := session.Query(`SELECT id, text FROM tweet WHERE timeline = ?`,
"me").Iter()
for iter.Scan(&id, &text) {
fmt.Println("Tweet:", id, text)
}
if err := iter.Close(); err != nil {
log.Fatal(err)
}
}
Using the Cassandra shell I have created the keyspace "example" and the table "tweet" and they all work fine.
However, when I run the program, it gives me this error:
2017/04/14 20:52:55 gocql: unable to dial control conn 192.168.1.3: dial tcp
192.168.1.3:9042: i/o timeout
gocql: unable to create session: control: unable to connect to initial
hosts: dial tcp 192.168.1.3:9042: i/o timeoutpanic: runtime error: invalid
memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x60236d]
goroutine 1 [running]:
github.com/gocql/gocql.(*Session).Close(0x0)
/home/mda/.local/go/src/github.com/gocql/gocql/session.go:344 +0x2d
For some reason, gocql is unable to dial the localhost connection, and it times out. I am not sure how to fix this and a stackoverflow and google search hasn't helped so far.
Any ideas?
Looks like the ip addresses are not according to the cluster situation on your computer. After giving me the nodetool status I think you should update the cluster := gocql.NewCluster to have just the 127.0.0.1 address and not the addresses from example.
I think in reality you are just running single node cassandra local instance. Which is fine for development.
Basically:
cluster := gocql.NewCluster("127.0.0.1")
and just drop this line all together
cluster.ProtoVersion = 3

How to pass (type *common.MapStr) to type []byte?

Sorry if the question is too newbie, as i just started to learn go yesterday.
I try to convert publishEvent into bytes, and compiler shown error like following:
cannot convert publishEvent (type *common.MapStr) to type []byte
Can anyone show me the way ?
Thank You.
var parsed map[string]interface{}
bytes := []byte(publishEvent) --->Error occur here
err := json.Unmarshal(bytes, &parsed)
if err != nil{
fmt.Println("error: ", err)
}
I assume the struct you are working with is common.MapStr from https://github.com/elastic/libbeat
common.MapStr is already a map[string]interface{} so I'm not sure why you are turing it into JSON, and then parsing it back into the same kind of structure, but if thats what you really want to do, replacing the error line with:
bytes, err := json.Marshal(publishEvent)
should work. You will get an error on the next line about redeclaring err so change it to:
err = json.Unmarshal(bytes, &parsed)
Resulting in the following code (also added another error check):
var parsed map[string]interface{}
bytes, err := json.Marshal(publishEvent)
if err != nil{
fmt.Println("error: ", err)
// you'll want to exit or return here since we can't parse `bytes`
}
err = json.Unmarshal(bytes, &parsed)
if err != nil{
fmt.Println("error: ", err)
}

Resources