I want to read a slice and append new data to it. I had a webapi in json format to read the statistics. So I want to grab the statistic, give it to a function, write a new slice with the existing stats from the json before.
Then I want to append new data at the end of it. And give the "new" Slice (Struct) with the old statistics from the json and the new together.
I tried so much, that's it.
This is the json looks like
{"24hreward":0,"currentHashrate":0,"hashrate":0,"history":[{"currenthashrate":6024360000,"online":0,"offline":0,"timestamp":0}],"pageSize":30,"payments":null,"paymentsTotal":0,"rewards":[{"blockheight":223115,"timestamp":1518179084,"blockhash":"0x*************ee03802ee52f411102159a7c9268fec4e46571daa07e84","reward":6024360000,"percent":1,"immature":false}],"roundShares":3674,"stats":{"balance":6024360000,"blocksFound":1,"immature":0,"lastShare":1518208862},"sumrewards":[{"inverval":3600,"reward":0,"name":"Last 60 minutes","offset":0},{"inverval":43200,"reward":0,"name":"Last 12 hours","offset":0},{"inverval":86400,"reward":0,"name":"Last 24 hours","offset":0},{"inverval":604800,"reward":0,"name":"Last 7 days","offset":0},{"inverval":2592000,"reward":6024360000,"name":"Last 30 days","offset":0}],"workers":{},"workersOffline":0,"workersOnline":0,"workersTotal":0}
That's the struct
type HistoryData struct {
CurrentHashrate int64 `json:"currenthashrate"`
Online int64 `json:"online"`
Offline int64 `json:"offline"`
Timestamp int64 `json:"timestamp"`
}
There you can see "history". I want to read the history and give it to my function here
cmds, err := tx.Exec(func() error {
tx.ZRemRangeByScore(r.formatKey("hashrate", login), "-inf", fmt.Sprint("(", now-largeWindow))
tx.ZRangeWithScores(r.formatKey("hashrate", login), 0, -1)
tx.LRange(r.formatKey("lastshares"), 0, 4999)
tx.ZRevRangeWithScores(r.formatKey("rewards", login), 0, 39)
tx.ZRevRangeWithScores(r.formatKey("rewards", login), 0, -1)
tx.ZRevRangeWithScores(r.formatKey("history", login), 0, 64)
return nil
})
stats["history"] = convertHistoryResults(currentHashrate, online, offline, now, cmds[5].(*redis.ZSliceCmd))
There I want to give in convertHistoryResults "currentHashrate, online, offline, now" and cmds[5] from top but there is nothing. The json shows data but he cannot read i think :/ Thats my Problem
And here the function convertHistoryResults to build the list for the "history" in the json
var count = 0;
func convertHistoryResults(currenthashrate int64, online int64, offline int64, now int64, rows ...*redis.ZSliceCmd) []*HistoryData {
var result []*HistoryData
log.Println("run1")
log.Println(count)
log.Println(rows)
history := HistoryData{}
for _, row := range rows {
for _, v := range row.Val() {
log.Println("run3")
fields := strings.Split(v.Member.(string), ":")
history.CurrentHashrate, _ = strconv.ParseInt(fields[0], 10, 64)
history.Online, _ = strconv.ParseInt(fields[1], 10, 64)
history.Offline, _ = strconv.ParseInt(fields[2], 10, 64)
history.Timestamp, _ = strconv.ParseInt(fields[3], 10, 64)
result = append(result, &history)
}
}
if(count > 0) { // Count = 40 ~4-6min
history.CurrentHashrate = currenthashrate
history.Online = online
history.Offline = offline
history.Timestamp = now
result = append(result, &history)
log.Println("run2")
//if len(result) > 300 {
// result = result[1:]
//}
count = 0;
} else {
log.Println("run4")
count++
}
return result
}
Here I build new HistoryData and want to read the "old" rows and append them to the HistoryData, than i want to give the new Data into it with count, cause i dont want this all seconds.
But my Problem is I cannot read the old "history" from the json but I don't know why.
With "rewards" its working. Its a code from https://github.com/sammy007/open-ethereum-pool
What i tried is, to read the "history" and give them to convertHistoryResults to build new History to append that to "history".
Hope someone can help me to get the old "history" and rebuild to new "history"
This script is running from much different users, so I must rebuild the "history" for each one running it.
That is the print in log
[ZREVRANGE miner:history:0x******************** 0 64 WITHSCORES: []]
You see, WITHSCORES is []
I also tried in for Rewards, thats like that
[ZREVRANGE miner:rewards:0x******************** 0 64 WITHSCORES: [{1.518179084e+09 6024360000:1.000000000:0:0x**********************159a7c9268fec4e46571daa07e84:223115:1518179084}]]
Related
According to this issue, the protoreflect package provides APIs for accessing the "unknown fields" of protobuf messages, but I don't see any way to use it if there isn't any existing schema. Basically, I want to perform a "weak decode", similar to what the JSON unmarshaller does if the output is of type map[string]interface{}.
The example from the documentation looks like this:
err := UnmarshalOptions{DiscardUnknown: true}.Unmarshal(b, m)
where b is the input byte slice and m is the output message, which needs to be initialised somehow, as you can see here. I was thinking that dynamicpb can be used for this purpose, but it doesn't look possible without having an existing MessageDescriptor and that's where I got stuck...
I was able to achieve this using the low level protowire package. Here is a full example, where I extract two fields of type uint64 (which happen to be assigned field numbers 4 and 5 in the original schema):
import "google.golang.org/protobuf/encoding/protowire"
func getData(src []byte) (creationTime, expiryTime uint64, err error) {
remaining := src
for len(remaining) > 0 {
fieldNum, wireType, n := protowire.ConsumeTag(remaining)
if n < 0 {
return 0, 0, fmt.Errorf("failed to consume tag: %w", protowire.ParseError(n))
}
remaining = remaining[n:]
switch fieldNum {
case 4: // Expiry time
if wireType != protowire.VarintType {
return 0, 0, fmt.Errorf("unexpected type for expiry time field: %d", wireType)
}
expiryTime, n = protowire.ConsumeVarint(remaining)
case 5: // Creation time
if wireType != protowire.VarintType {
return 0, 0, fmt.Errorf("unexpected type for creation time field: %d", wireType)
}
creationTime, n = protowire.ConsumeVarint(remaining)
default:
n = protowire.ConsumeFieldValue(fieldNum, wireType, remaining)
}
if n < 0 {
return 0, 0, fmt.Errorf("failed to consume value for field %d: %w", fieldNum, protowire.ParseError(n))
}
remaining = remaining[n:]
}
return
}
I need to update value of fields in multiple rows.
I'm querying to get some of the database rows, but it doesn't work.
DB.Where("is_send = ?", "0").Find(&artists)
for _, artist := range artists {
if condition {
artist.IsSend = 1
... (more updatee)
DB.Save(&artist)
}
}
Change how you range it, by referring the below example:
for _, elem := range elems {
elem = new_val // Won't work, because elem is a copy of
// the value from elems
}
for i := range elems {
elems[i] = new_val // Works, because elems[i] deferences
// the pointer to the actual value in elems
}
Read: Gotchas
Also, if you're not modifying all fields, rather than using Save you can use Update as well. Refer: GORM CRUD's Interface UPDATE
How do I append output from a twitter search to the field Data in the SearchTwitterOutput{} struct.
Thanks!
I am using a twitter library to search twitter base on a query input. The search returns an array of strings(I believe), I am able to fmt.println the data but I need the data as a struct.
type SearchTwitterOutput struct {
Data string
}
func (SearchTwitter) execute(input SearchTwitterInput) (*SearchTwitterOutput, error) {
credentials := Credentials{
AccessToken: input.AccessToken,
AccessTokenSecret: input.AccessTokenSecret,
ConsumerKey: input.ConsumerKey,
ConsumerSecret: input.ConsumerSecret,
}
client, err := GetUserClient(&credentials)
if err != nil {
return nil, err
}
// search through the tweet and returns a
search, _ , err := client.Search.Tweets(&twitter.SearchTweetParams{
Query: input.Text,
})
if err != nil {
println("PANIC")
panic(err.Error())
return &SearchTwitterOutput{}, err
}
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
}
return &SearchTwitterOutput{
Data: "test", //data is a string for now it can be anything
}, nil
}
//Data field is a string type for now it can be anything
//I use "test" as a placeholder, bc IDK...
Result from fmt.Printf("Tweet %d - %s\n", k, v.Text):
Tweet 0 - You know I had to do it to them! #JennaJulien #Jenna_Marbles #juliensolomita #notjulen Got my first hydroflask ever…
Tweet 1 - RT #brenna_hinshaw: I was in J2 today and watched someone fill their hydroflask with vanilla soft serve... what starts here changes the wor…
Tweet 2 - I miss my hydroflask :(
This is my second week working with go and new to development. Any help would be great.
It doesn't look like the client is just returning you a slice of strings. The range syntax you're using (for k, v := range search.Statuses) returns two values for each iteration, the index in the slice (in this case k), and the object from the slice (in this case v). I don't know the type of search.Statuses - but I know that strings don't have a .Text field or method, which is how you're printing v currently.
To your question:
Is there any particular reason to return just a single struct with a Data field rather than directly returning the output of the twitter client?
Your function signature could look like this instead:
func (SearchTwitter) execute(input SearchTwitterInput) ([]<client response struct>, error)
And then you could operate on the text in those objects in wherever this function was called.
If you're dead-set on placing the data in your own struct, you could return a slice of them ([]*SearchTwitterOutput), in which case you could build a single SearchTwitterOutput in the for loop you're currently printing the tweets in and append it to the output list. That might look like this:
var output []*SearchTwitterOutput
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
output = append(output, &SearchTwitterOutput{
Data: v.Text,
})
}
return output, nil
But if your goal really is to return all of the results concatenated together and placed inside a single struct, I would suggest building a slice of strings (containing the text you want), and then joining them with the delimiter of your choosing. Then you could place the single output string in your return object, which might look something like this:
var outputStrings []string
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
outputStrings = append(outputStrings, v.Text)
}
output = strings.Join(outputStrings, ",")
return &SearchTwitterOutput{
Data: output,
}, nil
Though I would caution, it might be tricky to find a delimiter that will never show up in a tweet..
When the RSS feeds updates (it doesn't right now, just dummy data) the new items are appended to the "feed" slice. Over time this could mean that it contains millions of items, I don't want that.
So when there are more than 100 items in the slice it should delete items starting from the top (item 0). In this example I'm using an RSS file with ust 100 items so the sample code below should delete from the top after 50 items:
package main
import (
"fmt"
"github.com/SlyMarbo/rss"
"time"
)
var feed *rss.Feed
var directory = "./dump"
func main() {
for {
checkRSS()
// Check every minute if feed.Refresh has passed so it run's update()
time.Sleep(1 * time.Minute)
}
}
func checkRSS() (*rss.Feed, error) {
var err error
// If feed is still empty fetch it first so we can run update()
if feed == nil {
feed, err = rss.Fetch("http://cloud.dgier.nl/api.xml")
} else {
err = feed.Update()
}
length := len(feed.Items)
for key, value := range feed.Items {
fmt.Println(key, value.Title, value.Read)
if key >= 50 {
fmt.Println("Item key is > 50")
}
}
fmt.Printf("Current length: %d\n", length)
fmt.Printf("Refreshing at %s\n", feed.Refresh)
return feed, err
}
If the number of items in the feed grows over the limit, slice it:
length := len(feed.Items)
if length > limit {
feed.Items = feed.Items[length - limit:]
}
When the length is over the limit, the new length will be exactly limit.
You don't need a for loop there.
To achieve this you probably want to use subslicing. Say you want to remove x items from feed, you can simply do feed = feed[x:] which will yield all items after index x-1 and assign it back to the feed slice. If in your actual code you just want to remove the first item then it would be feed = feed[1:]
I'm working in Go 1.6 on Windows and trying to export a certificate container to a PFX (the ultimate goal here is to access an exportable private key from the certificate store).
I have opened a memory store and inserted a certificate into the store:
var storedCertCtx *syscall.CertContext
storeHandle, err := syscall.CertOpenStore(syscall.CERT_STORE_PROV_MEMORY, 0, 0, syscall.CERT_STORE_DEFER_CLOSE_UNTIL_LAST_FREE_FLAG, 0)
err = syscall.CertAddCertificateContextToStore(storeHandle, certenum, syscall.CERT_STORE_ADD_ALWAYS, &storedCertCtx)
Now I want to generate a PFX of that store. I have defined a struct for containing the data blob and want to use PFXExportCertStoreEx to get a PFX of the store:
var (
crypt32 = syscall.NewLazyDLL("crypt32.dll")
procPFXExportCertStoreEx = crypt32.NewProc("PFXExportCertStoreEx")
)
type CRYPTOAPI_BLOB struct {
DataSize uint32
Data *byte
}
var pfxBlob CRYPTOAPI_BLOB
err = PfxExportCertStore(storeHandle, &pfxBlob, syscall.StringToUTF16Ptr("MyPassword"), 0, 0)
syscall.Syscall6(procPFXExportCertStoreEx.Addr(), 5,
uintptr(storeHandle), //hStore
uintptr(unsafe.Pointer(&pfxBlob)), //*pPFX
uintptr(unsafe.Pointer(syscall.StringToUTF16Ptr("password"))), //szPassword
0, //*pvPara
0, //dwFlags
0)
And this half works.
DataSize is populated with what looks like an appropriate value (i.e. if I add more certificates to the store, it grows bigger), however Data is always <nil>.
Seeing as it's meant to be populated with a pointer, I have tried declaring it as *uintptr and uint32 (just to see if anything gets populated), but nothing. The value is always untouched (if I manually put junk data in there, the junk data stays after the syscall is executed).
Have I defined the struct incorrectly? There is precious few examples to go for getting this done in Go, but from what I can see from the numerous C examples, this should be working.
This is the expected behavior.
According to this: https://msdn.microsoft.com/en-us/library/windows/desktop/aa387313(v=vs.85).aspx, the pPFX struct requires a pre-allocated buffer, with the size in the cbData field, which will be updated with the size of the data copied in.
If the call is made with pbData equal to NULL, only the cbData field is updated to reflect the size needed for the output buffer.
JimB's answer is most certainly correct, but I want to add this for followup in case anyone else is going down this path. The actual code that I had to use to get the PFX file into CRYPTOAPI_BLOB was:
var (
crypt32 = syscall.NewLazyDLL("crypt32.dll")
procPFXExportCertStoreEx = crypt32.NewProc("PFXExportCertStoreEx")
procCryptMemAlloc = crypt32.NewProc("CryptMemAlloc")
procCryptMemFree = crypt32.NewProc("CryptMemFree")
)
type CRYPTOAPI_BLOB struct {
cbData uint32
pbData *byte
}
func (b *CRYPTOAPI_BLOB) ToByteArray() []byte {
d := make([]byte, b.cbData)
copy(d, (*[1 << 30]byte)(unsafe.Pointer(b.pbData))[:])
return d
}
func PfxExportCertStore(storeHandle syscall.Handle, password string, flags uint32) (returnData []byte, err error) {
var pfxBlob CRYPTOAPI_BLOB
r1, _, _ := syscall.Syscall6(procPFXExportCertStoreEx.Addr(), 5,
uintptr(storeHandle), //hStore
uintptr(unsafe.Pointer(&pfxBlob)), //*pPFX
uintptr(unsafe.Pointer(syscall.StringToUTF16Ptr(password))), //szPassword
0, //*pvPara
uintptr(flags), //dwFlags
0)
r2, _, _ := syscall.Syscall(procCryptMemAlloc.Addr(), 1, uintptr(unsafe.Pointer(&pfxBlob.cbData)), 0, 0)
p := unsafe.Pointer(&r2)
q := (*byte)(p)
pfxBlob.pbData = q
defer syscall.Syscall(procCryptMemFree.Addr(), 1, uintptr(unsafe.Pointer(pfxBlob.pbData)), 0, 0)
r3, _, _ := syscall.Syscall6(procPFXExportCertStoreEx.Addr(), 5,
uintptr(storeHandle), //hStore
uintptr(unsafe.Pointer(&pfxBlob)), //*pPFX
uintptr(unsafe.Pointer(syscall.StringToUTF16Ptr(password))), //szPassword
0, //*pvPara
uintptr(flags), //dwFlags
0)
returnData = pfxBlob.ToByteArray()
return
}
(I have stripped the error handling to make it easier to read). The first call to PFXExportCertStoreEx just returns the size, and once we have the size we can do a call to PFXExportCertStoreEx to allocate a buffer, and then we pass the same pointer to PFXExportCertStoreEx, but this time it has the allocated buffer, and we get the full PFX file returned.