How to save multi JSON API response to MongoDB using Golang - go

I am fetching data from an api which comes in this format (10+ individual orders):
{
"orders": [
{
"orderId": "2738183220",
"dateTimeOrderPlaced": "2019-12-11T20:19:19+01:00",
"orderItems": [
{
"orderItemId": "BFC0000347027892",
"ean": "8720165489992",
"cancelRequest": false,
"quantity": 1
}
]
},
{
"orderId": "2737526230",
"dateTimeOrderPlaced": "2019-12-11T17:18:42+01:00",
"orderItems": [
{
"orderItemId": "BFC0000346926381",
"ean": "8719326639581",
"cancelRequest": false,
"quantity": 1
}
]
}
]
}
I have this struct for that:
type OpenOrder struct {
Orders []struct {
OrderID string `json:"orderId"`
DateTimeOrderPlaced time.Time `json:"dateTimeOrderPlaced"`
OrderItems []struct {
OrderItemID string `json:"orderItemId"`
Ean string `json:"ean"`
CancelRequest bool `json:"cancelRequest"`
Quantity int `json:"quantity"`
} `json:"orderItems"`
} `json:"orders"`
}
But I am confused after looking at this for too long. How can I save this into MongoDB with Golang where so it will get saved as single documents? So one document is:
{
"orderId": "2738183220",
"dateTimeOrderPlaced": "2019-12-11T20:19:19+01:00",
"orderItems": [
{
"orderItemId": "BFC0000347027892",
"ean": "8720165489992",
"cancelRequest": false,
"quantity": 1
}
]
}
I wrote this code:
func GetOpenOrder(x string) {
apiURL := "https://api.bol.com/retailer/orders?fulfilment-method=" + x
req, err := http.NewRequest("GET", apiURL, nil)
req.Header.Add("Authorization", "Bearer "+auth.GetToken())
req.Header.Add("Accept", "application/vnd.retailer.v3+json")
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
panic(err)
}
// Read the response body
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
}
err = json.Unmarshal(body, &o)
if err != nil {
fmt.Println(string(body))
}
c := auth.GetClient()
collection := c.Database("goprac").Collection("openorders")
ctx, _ := context.WithTimeout(context.Background(), 5*time.Second)
result, _ := collection.InsertOne(ctx, o)
fmt.Println(result)
}
But right now the whole JSON response gets saved in one document. Can someone please point me in the correct direction? I have been searching SO for the last hour and found some similar questions, but still did not get the answer.

Related

How to unmarshal/parse a request using Go correctly?

How to parse a curl request response in Golang correctly? I have tried the following where I send a request to an api and its response is:
{
"Certificates": [
{
"Name": "some-name.here",
.......
}
],
"DataRange": "Certificates 1 - 1",
"TotalCount": 1
}
Now I want to use the Name in the Certificates in a string variable. i.e match. Before even I get to the looping through of the response, I get the error: json: cannot unmarshal object into Go value of type []program.Item. This error is coming from json.Unmarshal function where I pass my []bytes and the struct to use the bytes. Am I doing this correctly?
type Item struct {
Certificates []CertificatesInfo
}
type CertificatesInfo struct {
Name string
}
func main() {
url := .....
req, err := http.NewRequest("GET", url, nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
// handle err
continue
}
defer resp.Body.Close()
bodyBytes, err := io.ReadAll(resp.Body)
if err != nil {
continue
}
var ItemInfo []Item
if err := json.Unmarshal(bodyBytes, &ItemInfo); err != nil {
return nil, fmt.Errorf("failed to parse %v", err)
}
for _, infos := range ItemInfo {
for _, names := range infos.Certificates {
infraId:= names.Name
match:= display_name
}
}
}
There's a mismatch between your sample JSON response data (a single object containing a list of certificates), and your code, which is expecting a list of objects, each of which contain a list of certificates).
Assuming that your JSON example is correct, this bit:
var ItemInfo []Item
if err := json.Unmarshal(bodyBytes, &ItemInfo); err != nil {
. . .
should probably be
var itemInfo Item
if err := json.Unmarshal(bodyBytes, &ItemInfo); err != nil {
. . .
Works on my machine:
https://go.dev/play/p/8FqIif1zzsQ
package main
import (
"encoding/json"
"fmt"
)
func main() {
resp := Item{}
err := json.Unmarshal([]byte(data), &resp)
if err != nil {
fmt.Print(err)
panic(err)
}
fmt.Printf("resp.DataRange: %s\n", resp.DataRange)
fmt.Printf("resp.TotalCount: %d\n", resp.TotalCount)
fmt.Printf("len(resp.Certificates: %d\n", len(resp.Certificates))
for i, c := range resp.Certificates {
fmt.Printf("resp.Certificates[%d].Name: %s\n", i, c.Name)
}
}
type Item struct {
Certificates []CertificatesInfo
DataRange string
TotalCount int
}
type CertificatesInfo struct {
Name string
}
const data = `
{
"Certificates": [
{ "Name": "alpha" },
{ "Name": "bravo" },
{ "Name": "charlie" }
],
"DataRange": "Certificates 1 - 1",
"TotalCount": 3
}
`

Cannot broadcast bitcoin transaction Golang

I've encouraged an error broadcasting my bitcoin transaction here.
Error validating transaction: Error running script for input 0 referencing 78b6df775e4132f747d8d15909cfe61834e6ff1d3b1e42219ace64aae7151e18 at 0: Script was NOT verified successfully..
I've tried to put decoded Asm PkScript instead of PubKey in TxIn. I've tried to find similar error on sof but their steps didn't work for me. Can anyone find out what is wrong with this tx? because my vm didn't find any troubles.
Hex transaction:
0100000001181e15e7aa64ce9a21421e3b1dffe63418e6cf0959d1d847f732415e77dfb678000000008a47304402200aa339f80c705a22180c8edd4acc4baa5bc76c9e8f64852eb3cb507f027aee9c02206da34700e76df568ef018fe346bb5b9c4484506ee84740916d6c3b454c2e1d3f0141042a1b4d9533ff37cd290b95016056cacbdbbf99b5975345dc5837a95f698f8a26d74f726e2a1e62552107953595502c8078a84311690c61ace3f22644ae33a0abffffffff02d0070000000000001976a91437383464376cccaf2ae4c8a121805f45bf544e4488acc8320000000000001976a914a404271e971d58240add530813e514b0ffe45f3588ac00000000
Decoded transaction:
{
"addresses": [
"mvUC64ETwYLbTjH38ZVigDwMBtCS9TbuT1",
"mkYvnmm3KUBkvVqUAYsG6A6amt5Dva4jzX"
],
"block_height": -1,
"block_index": -1,
"confirmations": 0,
"double_spend": false,
"fees": 5000,
"hash": "410f8119e9c225fc197a21ede7f9e6902f85a36010a2776dd3fdc2663072a36c",
"inputs": [
{
"addresses": [
"mvUC64ETwYLbTjH38ZVigDwMBtCS9TbuT1"
],
"age": 2345496,
"output_index": 0,
"output_value": 20000,
"prev_hash": "78b6df775e4132f747d8d15909cfe61834e6ff1d3b1e42219ace64aae7151e18",
"script": "47304402200aa339f80c705a22180c8edd4acc4baa5bc76c9e8f64852eb3cb507f027aee9c02206da34700e76df568ef018fe346bb5b9c4484506ee84740916d6c3b454c2e1d3f0141042a1b4d9533ff37cd290b95016056cacbdbbf99b5975345dc5837a95f698f8a26d74f726e2a1e62552107953595502c8078a84311690c61ace3f22644ae33a0ab",
"script_type": "pay-to-pubkey-hash",
"sequence": 4294967295
}
],
"outputs": [
{
"addresses": [
"mkYvnmm3KUBkvVqUAYsG6A6amt5Dva4jzX"
],
"script": "76a91437383464376cccaf2ae4c8a121805f45bf544e4488ac",
"script_type": "pay-to-pubkey-hash",
"value": 2000
},
{
"addresses": [
"mvUC64ETwYLbTjH38ZVigDwMBtCS9TbuT1"
],
"script": "76a914a404271e971d58240add530813e514b0ffe45f3588ac",
"script_type": "pay-to-pubkey-hash",
"value": 13000
}
],
"preference": "low",
"received": "2022-09-06T14:10:50.749940874Z",
"relayed_by": "44.204.10.232",
"size": 257,
"total": 15000,
"ver": 1,
"vin_sz": 1,
"vout_sz": 2,
"vsize": 257
}
func main() {
connCfg := &rpcclient.ConnConfig{ //smth here
}
client, err := rpcclient.New(connCfg, nil)
if err != nil {
log.Fatal(err)
}
defer client.Shutdown()
utxoMap := make(map[string]string)
utxoMap["mvUC64ETwYLbTjH38ZVigDwMBtCS9TbuT1"] = "78b6df775e4132f747d8d15909cfe61834e6ff1d3b1e42219ace64aae7151e18"
rawTx, err := CreateRawTx("mvUC64ETwYLbTjH38ZVigDwMBtCS9TbuT1", "mkYvnmm3KUBkvVqUAYsG6A6amt5Dva4jzX", 2000, utxoMap, client)
if err != nil {
println(err.Error())
}
signedTx, err := Sign(rawTx, "private key here")
if err != nil {
println(err.Error())
}
fmt.Println(signedTx)
}
type UTXO struct {
previousTxID string
pubKeyScript string
Asm string
Amount decimal.Decimal
}
func GetUTXO(utxoMap map[string]string, address string, client *rpcclient.Client) (utxo UTXO, err error) {
// unmarshal it, and extract necessary data
hash, _ := chainhash.NewHashFromStr(utxoMap[address])
tx, err := client.GetRawTransactionVerbose(hash)
if err != nil {
return utxo, errors.Wrapf(err, "GetUTXO.GetRawTransactionVerbose")
}
pubKeyScript := tx.Vout[0].ScriptPubKey.Hex
previousTxID := utxoMap[address]
amount := decimal.NewFromFloat(tx.Vout[0].Value)
asm := tx.Vout[0].ScriptPubKey.Hex
return UTXO{
previousTxID: previousTxID,
pubKeyScript: pubKeyScript,
Asm: asm,
Amount: amount,
}, nil
}
func GetPayToAddrScript(address string) []byte {
rcvAddress, _ := btcutil.DecodeAddress(address, &chaincfg.TestNet3Params)
rcvScript, _ := txscript.PayToAddrScript(rcvAddress)
return rcvScript
}
type TXRef struct {
TXHash string
TXOutputN int
}
// get private key
func getKeyAddressFromPrivateKey(privateKey string, conf *chaincfg.Params) (*btcec.PrivateKey, string, error) {
newWif, err := btcutil.DecodeWIF(privateKey)
if err != nil {
return nil, "", err
}
publicKey := newWif.PrivKey.PubKey().SerializeUncompressed()
address, err := btcutil.NewAddressPubKeyHash(btcutil.Hash160(publicKey), conf)
if err != nil {
return nil, "", err
}
return newWif.PrivKey, address.EncodeAddress(), nil
}
// debug signing function
func Sign(rawTx, privateKey string) (string, error) {
var signTx wire.MsgTx
err := json.Unmarshal([]byte(rawTx), &signTx)
if err != nil {
return "", err
}
myPrivateKey, address, err := getKeyAddressFromPrivateKey(privateKey, &chaincfg.TestNet3Params)
fmt.Println("address ", address)
if err != nil {
return "", err
}
txScript := GetPayToAddrScript(address)
for i := 0; i < len(signTx.TxIn); i++ {
sig, err := txscript.SignatureScript(
&signTx, // The tx to be signed.
i, // The index of the txin the signature is for.
txScript, // The other half of the script from the PubKeyHash.
txscript.SigHashAll, // The signature flags that indicate what the sig covers.
myPrivateKey, // The key to generate the signature with.
false) // The compress sig flag. This saves space on the blockchain.
if err != nil {
return "", err
}
//refer from this link for sign multiple inputs https://bitcoin.stackexchange.com/questions/41209/how-to-sign-a-transaction-with-multiple-inputs
signTx.TxIn[i].SignatureScript = sig
//Validate signature
flags := txscript.StandardVerifyFlags
vm, err := txscript.NewEngine(txScript, &signTx, i, flags, nil, nil, signTx.TxOut[0].Value, nil)
if err != nil {
return "", err
}
if err := vm.Execute(); err != nil {
return "", err
}
}
// return signed tx
var signedTx bytes.Buffer
signTx.Serialize(&signedTx)
hexSignedTx := hex.EncodeToString(signedTx.Bytes())
return hexSignedTx, nil
}
// CreateBtcRawTx create raw tx for sending btc
func CreateRawTx(from string, to string, amount int64, utxoMap map[string]string, client *rpcclient.Client) (string, error) {
amountInt := amount
var feeBitcoin int64 = 5000
spendAmount := amountInt + feeBitcoin
utxo, err := GetUTXO(utxoMap, from, client)
changeAmount := utxo.Amount.Shift(8).IntPart() - spendAmount
// create new empty transaction
redemTx := wire.NewMsgTx(wire.TxVersion)
// create multiple txIns
hash, err := chainhash.NewHashFromStr(utxo.previousTxID)
if err != nil {
fmt.Printf("could not get hash from transaction ID: %v", err)
return "", err
}
outPoint := wire.NewOutPoint(hash, 0)
txIn := wire.NewTxIn(outPoint, nil, nil)
redemTx.AddTxIn(txIn)
// create TxOut
rcvScript := GetPayToAddrScript(to)
txOut := wire.NewTxOut(amountInt, rcvScript)
redemTx.AddTxOut(txOut)
// create TxOut for change address, in this case, change address is sender itself
if changeAmount > 0 {
// return change BTC to its own address
rcvChangeAddressScript := GetPayToAddrScript(from)
txOut := wire.NewTxOut(changeAmount, rcvChangeAddressScript)
redemTx.AddTxOut(txOut)
}
encodedTx, err := json.Marshal(redemTx)
if err != nil {
return "", err
}
// return raw tx in hex format
return string(encodedTx), nil
}
my bitcoin wallet: mvUC64ETwYLbTjH38ZVigDwMBtCS9TbuT1
last transaction on the wallet (hash): 78b6df775e4132f747d8d15909cfe61834e6ff1d3b1e42219ace64aae7151e18

interface conversion: error is *errors.errorString, not validator.ValidationErrors

type BookInput struct {
Title string `json:"title" binding:"required"`
Price json.Number `json:"price" binding:"required,number"`
}
func PostBookHandler(ctx *gin.Context) {
var bookInput book.BookInput
err := ctx.ShouldBindJSON(&bookInput)
if err != nil {
errorMessages := []string{}
for _, e := range err.(validator.ValidationErrors) {
errorMessage := fmt.Sprintf("Error on filed %s, condition: %s", e.Field(), e.ActualTag())
errorMessages = append(errorMessages, errorMessage)
}
ctx.JSON(http.StatusBadRequest, gin.H {
"errors": errorMessages,
})
return
}
ctx.JSON(http.StatusOK, gin.H {
"title": bookInput.Title,
"price": bookInput.Price,
})
}
I tried to validate the price input, but the results I got were unexpected. The code I wrote is like the one above, can someone help me?
The error returned in this case might not be a validator.ValidationErrors, it could be something else. For example if the body is invalid JSON, then the validation step isn’t reached at all.
In your code you are performing an unchecked assertion range err.(validator.ValidationErrors) which could panic.
This is how you could conditionally handle the error:
err := ctx.ShouldBindJSON(&bookInput)
if err != nil {
var ve validator.ValidationErrors
if errors.As(err, &ve) {
// handle validator error
}
// handle non-validator error
return
}
Maybe it can help:
type BookInput struct {
Title string `json:"title" binding:"required"`
Price interface{} `json:"price" binding:"required,number"`
}
func postBooksHandler(c *gin.Context) {
var bookInput BookInput
err := c.ShouldBindJSON(&bookInput)
if err != nil {
errorMessages := []string{}
for _, e := range err.(validator.ValidationErrors) {
errorMessage := fmt.Sprintf("Error on field %s, conditon: %s", e.Field(), e.ActualTag())
errorMessages = append(errorMessages, errorMessage)
}
c.JSON(http.StatusBadRequest, gin.H{
"error": errorMessages,
})
return
}
c.JSON(http.StatusOK, gin.H{
"title": bookInput.Title,
"price": bookInput.Price,
})
}
Need to create two conditions on error, because validator.ValidationErrors doesn't cover everything.
This my code:
if err != nil {
var ve validator.ValidationErrors
if errors.As(err, &ve) {
for _, e := range err.(validator.ValidationErrors) {
c.JSON(http.StatusBadRequest, gin.H{
"error": true,
"message": "" + e.Field() + " kosong",
})
return
}
}
c.JSON(http.StatusBadRequest, gin.H{
"error": true,
"message": err.Error(),
})
}

how to fetch data from another api with body raw json

i have default body raw json and want to paste it into a struct so it can fetch data automatically and save it into a struct
Body Raw Json
{
"jsonrpc": "2.0",
"params": {
}
}
Response from api
{
"jsonrpc": "2.0",
"id": null,
"result": {
"status": 200,
"response": [
{
"service_id": 1129,
"service_name": "Adobe Illustrator",
"service_category_id": 28,
"service_category_name": "License Software",
"service_type_id": 25,
"service_type_name": "Software",
"create_date": "2020-03-09 03:47:44"
},
],
"message": "Done All User Returned"
}
}
I want to put it in the repository file so I can get data automatically
Repo file
// Get request
resp, err := http.Get("look at API Response Example")
if err != nil {
fmt.Println("No response from request")
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body) // response body is []byte
if err != nil {
return err
}
// data that already fetch accomadate to struct
var result models.OdooRequest
if err := json.Unmarshal(body, &result); err != nil {
fmt.Println("Can not unmarshal JSON")
}
for _, rec := range result.Response {
fmt.Println(rec.ServiceName)
}
return err
after being fetched then accommodated into a struct
struct
type OdooRequest struct {
Response []UpsertFromOdooServices
}
Sure, here's a rough way to make that request and read the response:
package main
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
)
type OdooRequest struct {
Result struct {
Status int `json:"status"`
Response []struct {
ServiceID int `json:"service_id"`
ServiceName string `json:"service_name"`
ServiceCategoryID int `json:"service_category_id"`
ServiceCategoryName string `json:"service_category_name"`
ServiceTypeID int `json:"service_type_id"`
ServiceTypeName string `json:"service_type_name"`
CreateDate string `json:"create_date"`
} `json:"response"`
Message string `json:"message"`
} `json:"result"`
}
func main() {
if err := run(); err != nil {
panic(err)
}
}
func run() error {
resp, err := http.Post(
"ADD_URL_HERE",
"application/json",
bytes.NewBufferString(`{"jsonrpc": "2.0","params": {}}`),
)
if err != nil {
return err
}
defer resp.Body.Close()
var odooResp OdooRequest
if err := json.NewDecoder(resp.Body).Decode(&odooResp); err != nil {
return err
}
for _, rec := range odooResp.Result.Response {
fmt.Println(rec.ServiceName)
}
return nil
}

How to return valid jsonapi response using google/jsonapi and echo framework

The code bellow return a two concated JSON strings and a wrong content-type text/plain. Should be application/vnd.api+json
package main
import (
"github.com/google/jsonapi"
"github.com/labstack/echo"
"net/http"
)
type Album struct {
ID int `jsonapi:"primary,albums"`
Name string `jsonapi:"attr,name"`
}
func main() {
e := echo.New()
e.GET("/", func(c echo.Context) error {
jsonapi.MarshalManyPayload(c.Response(), albumList())
return c.JSON(http.StatusOK, c.Response())
})
e.Logger.Fatal(e.Start(":1323"))
}
func albumList() []*Album {
a1 := Album{123, "allbum1"}
a2 := Album{456, "allbum2"}
albums := []*Album{&a1, &a2}
return albums
}
faulty output (two concated jsons). The first is a correct jsonapi structure and I think the second is related to echo-framework:
{
"data": [
{
"type": "albums",
"id": "123",
"attributes": {
"name": "allbum1"
}
},
{
"type": "albums",
"id": "456",
"attributes": {
"name": "allbum2"
}
}
]
}
{
"Writer": {},
"Status": 200,
"Size": 133,
"Committed": true
}
This code fix the problem but is seems awkward. I have the feeling there is a better way to facilitate it using echo.
e.GET("/", func(c echo.Context) error {
var b bytes.Buffer
body := bufio.NewWriter(&b)
err := jsonapi.MarshalManyPayload(body, albumList())
if err != nil {
fmt.Println(err)
}
body.Flush()
return c.JSONBlob(http.StatusOK, b.Bytes())
})
Any idea?
You're code looks alright. However it can be simplified-
var b bytes.Buffer // you could use buffer pool here
err := jsonapi.MarshalManyPayload(&b, albumList())
if err != nil {
return err
}
return c.JSONBlob(http.StatusOK, b.Bytes())
Following approaches for your thoughts:
Approach 1 -
c.Response().Header().Set(echo.HeaderContentType, jsonapi.MediaType)
c.Response().WriteHeader(http.StatusOK)
return jsonapi.MarshalManyPayload(c.Response(), albumList())
Approach 2 -
var b bytes.Buffer // you could use buffer pool here
err := jsonapi.MarshalManyPayload(&b, albumList())
if err != nil {
return err
}
c.Response().Header().Set(echo.HeaderContentType, jsonapi.MediaType)
c.Response().WriteHeader(http.StatusOK)
_, err := b.WriteTo(c.Response())
return err

Resources