I want to use the golang library https://github.com/anacrolix/torrent to create to torrent and get a magnet and seed the torrent. Below you can find the code I wrote. Yet, if I use the magnet the code generates I can not download anything not even the metainfo.
Am I missing something here?
package main
import (
"log"
"time"
"github.com/anacrolix/torrent"
"github.com/anacrolix/torrent/bencode"
"github.com/anacrolix/torrent/metainfo"
)
var builtinAnnounceList = [][]string{
{"http://p4p.arenabg.com:1337/announce"},
{"udp://tracker.opentrackr.org:1337/announce"},
{"udp://tracker.openbittorrent.com:6969/announce"},
}
func main() {
tmpComment:="Cool torrent description"
tmpCreatedBy:="coolboys"
tmpInfoName:="CoolInfoName"
mi := metainfo.MetaInfo{
AnnounceList: builtinAnnounceList,
}
mi.SetDefaults()
mi.Comment = tmpComment
mi.CreatedBy = tmpCreatedBy
//}
//mi.UrlList = args.Url//???????????
info := metainfo.Info{
PieceLength: 256 * 1024,
}
err := info.BuildFromFilePath("./TorrentFiles")//args.Root)
if err != nil {
log.Fatal(err)
}
info.Name =tmpInfoName
mi.InfoBytes, err = bencode.Marshal(info)
if err != nil {
log.Fatal(err)
}
tmpMagnet:=mi.Magnet(nil,nil)
log.Println("****",tmpMagnet)
//
cfg := torrent.NewDefaultClientConfig()
cfg.Seed = true
mainclient, ncerr := torrent.NewClient(cfg)
if ncerr != nil {
log.Println("new torrent client:", ncerr)
return
}
defer mainclient.Close()
t, _ := mainclient.AddMagnet(tmpMagnet.String())
for {
log.Println("******", t.Seeding())
time.Sleep(8 * time.Second)
}
}
I think you might need to take a closer look at this example. Among other things, I don't see any invocation of t.DownloadAll() to do the actual download or mainclient.WaitAll() to tell you when the downloads are complete.
I corrected the code I wrote and the correct code can be found here:
enter link description here
In the original code I should not have used addmagnet as it assumes that I don't have the info available, which is why it would fail to seed.
Related
Since a few days, I'm trying to download file from Huawei, and more precisely on their cloud storage. The issue is, I haven't been able to connect to it...
I found a SDK from huawei : https://github.com/huaweicloud/huaweicloud-sdk-go-v3
But I'm a little bit lost on all the protocol I can use to connect to it, and I haven't been able to make one working, each time. And to be honest, documentations isn't really helping me...
I also found this : https://github.com/huaweicloud/huaweicloud-sdk-go-obs
There is an example of downloading a file. Here, I can't even connect to Huawei... In the AppGalery, project settings, I downloaded the configuration file and tried the endpoint, but without success...
Here is what I tried with obs (I know/guess it should be agc, but I haven't found a package for it), but not working due to the host...
/**
* This sample demonstrates how to download an object
* from OBS in different ways using the OBS SDK for Go.
*/
package huawei
import (
"fmt"
"io"
"github.com/huaweicloud/huaweicloud-sdk-go-obs/obs"
)
type DownloadSample struct {
bucketName string
objectKey string
location string
obsClient *obs.ObsClient
}
func newDownloadSample(ak, sk, endpoint, bucketName, objectKey, location string) *DownloadSample {
obsClient, err := obs.New(ak, sk, endpoint)
if err != nil {
panic(err)
}
return &DownloadSample{obsClient: obsClient, bucketName: bucketName, objectKey: objectKey, location: location}
}
func (sample DownloadSample) GetObject() {
input := &obs.GetObjectInput{}
input.Bucket = sample.bucketName
input.Key = sample.objectKey
fmt.Printf("%+v\n", input)
output, err := sample.obsClient.GetObject(input)
if err != nil {
panic(err)
}
defer func() {
errMsg := output.Body.Close()
if errMsg != nil {
panic(errMsg)
}
}()
fmt.Println("Object content:")
body, err := io.ReadAll(output.Body)
if err != nil {
panic(err)
}
fmt.Println(string(body))
fmt.Println()
}
func RunDownloadSample() {
const (
endpoint = "theEndPointInConfigJSONFile"
ak = "prettySureOfThis"
sk = "prettySureOfThis"
bucketName = "prettySureOfThis"
objectKey = "test.txt" // a txt in the bucket to try to download it
location = ""
)
sample := newDownloadSample(ak, sk, endpoint, bucketName, objectKey, location)
fmt.Println("Download object to string")
sample.GetObject()
}
Thank you for your help
I am calling this code:
package multicall
import (
"fmt"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/ethclient"
"github.com/ethereum/go-ethereum/accounts/abi/bind"
)
var multicallContractAddress = common.HexToAddress("0x5e227AD1969Ea493B43F840cfF78d08a6fc17796")
var multicallContractEthBalanceSelector = "4d2301cc"
func GetBalances(addresses []string, ETHProviderURL string) ([]string, error) {
ethProvider, err := ethclient.Dial(ETHProviderURL)
if err != nil {
panic(err)
}
multicallContract, err := NewMulticallCaller(multicallContractAddress, ethProvider)
if err != nil {
panic(err)
}
fmt.Println("multicallContract: ", multicallContract)
var calls = []MulticallCall{}
for _, address := range addresses {
hashAddress := common.HexToHash(address)
call := MulticallCall{multicallContractAddress, []byte("0x"+multicallContractEthBalanceSelector+hashAddress.String()[2:])}
calls = append(calls, call)
}
fmt.Println(string(calls[0].CallData))
var results []byte
err = multicallContract.contract.Call(&bind.CallOpts{}, results, "aggregate", common.Hex2Bytes(addresses[0]))
if err != nil {
panic(err)
}
fmt.Println("Result: ", &results)
return nil, nil
}
like this:
multicall.GetBalances([]string{"<MY_ADDRESS>"}, "<INFURA_API_KEY>")
And it returns this error:
# github.com/mteam88/keyswarm/multicall
multicall/multicall.go:37:58: cannot use results (variable of type []byte) as type *[]interface{} in argument to multicallContract.contract.Call
It seems like results should be a specific type/interface (maybe []uint8) but I can't seem to pass it like that to the Call function.
I have tried tons of things and I can't seem to get it. My Google diving has uncovered this Why am I getting a compile error 'cannot use ... as type uint8 in argument to ...' when the parameter is an int which led me to my previous assumption.
If you -1 please comment (:
Please provide links to relevant documentation in your answer.
NOTE: https://github.com/mteam88/keyswarm/blob/multicall/multicall/multicallContract.go is the location of the defenitions of MulticallCall and other Multicall prefixed defenitions.
I have this module that use Google Cloud API to retrieve a list of all running Virtual Machine instances for a particular project. I'm new to Go, and followed the intro tutorial to help me out. I'm still trying to debug my code but no luck.
The problem is I'm able to communicate to Google Cloud API and pass authentication but that is all I can get through
compute.go module:
compute.go is able to communicate to Google Cloud servers and pass authentication (I'm not getting an auth error)
// Copyright 2021 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package compute
// [START compute_instances_list_all]
import (
"context"
"fmt"
"io"
compute "cloud.google.com/go/compute/apiv1"
"google.golang.org/api/iterator"
computepb "google.golang.org/genproto/googleapis/cloud/compute/v1"
"google.golang.org/protobuf/proto"
)
// listAllInstances prints all instances present in a project, grouped by their zone.
func ListAllInstances(w io.Writer, projectID string) error {
// projectID := "your_project_id"
ctx := context.Background()
instancesClient, err := compute.NewInstancesRESTClient(ctx)
// instancesClient, err := compute.NewInstancesRESTClient(ctx, option.WithCredentialsFile(`C:\path\to\jsonkey.json`))
if err != nil {
return fmt.Errorf("NewInstancesRESTClient: %v", err)
}
defer instancesClient.Close()
// Use the `MaxResults` parameter to limit the number of results that the API returns per response page.
req := &computepb.AggregatedListInstancesRequest{
Project: projectID,
MaxResults: proto.Uint32(6),
}
it := instancesClient.AggregatedList(ctx, req)
fmt.Fprintf(w, "Instances found:\n")
// Despite using the `MaxResults` parameter, you don't need to handle the pagination
// yourself. The returned iterator object handles pagination
// automatically, returning separated pages as you iterate over the results.
for {
pair, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return err
}
instances := pair.Value.Instances
if len(instances) > 0 {
fmt.Fprintf(w, "%s\n", pair.Key)
for _, instance := range instances {
fmt.Fprintf(w, "- %s %s\n", instance.GetName(), instance.GetMachineType())
}
}
}
return nil
}
// [END compute_instances_list_all]
However the problem is when I run my main function that calls ListAllInstances, it returns a <nil>. Not allowing me to know what is wrong.
caller api.go module where I run go run .:
package main
import (
"fmt"
"example.com/compute"
"bytes"
)
func main() {
buf := new(bytes.Buffer)
// Get a message and print it.
respone := compute.ListAllInstances(buf, "project-unique-id")
fmt.Println(respone)
}
How else can I further debug this to figure out what is wrong with my code?
You're not printing buf. Your function returns an object of type error, which is nil (no error!), the actual output is written to buf.
Either print it out:
func main() {
buf := new(bytes.Buffer)
// Get a message and print it.
err := compute.ListAllInstances(buf, "project-unique-id")
if err != nil {
panic(err)
}
fmt.Println(buf.String()) // <======= Print buf contents!
}
Or just use os.Stdout:
func main() {
err := compute.ListAllInstances(os.Stdout, "project-unique-id")
if err != nil {
panic(err)
}
}
To answer your question about debugging, try using VSCode with the Go extension, in there you can run a debugger, set breakpoints and step through the code line-by-line, watching how variables change.
See also Debug Go programs in VS Code.
I've a go application that gets run periodically by a batch. Each run, it should read some prometheus metrics from a file, run its logic, update a success/fail counter, and write metrics back out to a file.
From looking at How to parse Prometheus data as well as the godocs for prometheus, I'm able to read in the file, but I don't know how to update app_processed_total with the value returned by expfmt.ExtractSamples().
This is what I've done so far. Could someone please tell me how should I proceed from here? How can I typecast the Vector I got into a CounterVec?
package main
import (
"fmt"
"net/http"
"strings"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
dto "github.com/prometheus/client_model/go"
"github.com/prometheus/common/expfmt"
"github.com/prometheus/common/model"
)
var (
fileOnDisk = prometheus.NewRegistry()
processedTotal = prometheus.NewCounterVec(prometheus.CounterOpts{
Name: "app_processed_total",
Help: "Number of times ran",
}, []string{"status"})
)
func doInit() {
prometheus.MustRegister(processedTotal)
}
func recordMetrics() {
go func() {
for {
processedTotal.With(prometheus.Labels{"status": "ok"}).Inc()
time.Sleep(5 * time.Second)
}
}()
}
func readExistingMetrics() {
var parser expfmt.TextParser
text := `
# HELP app_processed_total Number of times ran
# TYPE app_processed_total counter
app_processed_total{status="ok"} 300
`
parseText := func() ([]*dto.MetricFamily, error) {
parsed, err := parser.TextToMetricFamilies(strings.NewReader(text))
if err != nil {
return nil, err
}
var result []*dto.MetricFamily
for _, mf := range parsed {
result = append(result, mf)
}
return result, nil
}
gatherers := prometheus.Gatherers{
fileOnDisk,
prometheus.GathererFunc(parseText),
}
gathering, err := gatherers.Gather()
if err != nil {
fmt.Println(err)
}
fmt.Println("gathering: ", gathering)
for _, g := range gathering {
vector, err := expfmt.ExtractSamples(&expfmt.DecodeOptions{
Timestamp: model.Now(),
}, g)
fmt.Println("vector: ", vector)
if err != nil {
fmt.Println(err)
}
// How can I update processedTotal with this new value?
}
}
func main() {
doInit()
readExistingMetrics()
recordMetrics()
http.Handle("/metrics", promhttp.Handler())
http.ListenAndServe("localhost:2112", nil)
}
I believe you would need to use processedTotal.WithLabelValues("ok").Inc() or something similar to that.
The more complete example is here
func ExampleCounterVec() {
httpReqs := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "http_requests_total",
Help: "How many HTTP requests processed, partitioned by status code and HTTP method.",
},
[]string{"code", "method"},
)
prometheus.MustRegister(httpReqs)
httpReqs.WithLabelValues("404", "POST").Add(42)
// If you have to access the same set of labels very frequently, it
// might be good to retrieve the metric only once and keep a handle to
// it. But beware of deletion of that metric, see below!
m := httpReqs.WithLabelValues("200", "GET")
for i := 0; i < 1000000; i++ {
m.Inc()
}
// Delete a metric from the vector. If you have previously kept a handle
// to that metric (as above), future updates via that handle will go
// unseen (even if you re-create a metric with the same label set
// later).
httpReqs.DeleteLabelValues("200", "GET")
// Same thing with the more verbose Labels syntax.
httpReqs.Delete(prometheus.Labels{"method": "GET", "code": "200"})
}
This is taken from the Promethus examples on Github
To use the value of vector you can do the following:
vectorFloat, err := strconv.ParseFloat(vector[0].Value.String(), 64)
if err != nil {
panic(err)
}
processedTotal.WithLabelValues("ok").Add(vectorFloat)
This is assuming you will only ever get a single vector value in your response. The value of the vector is stored as a string but you can convert it to a float with the strconv.ParseFloat method.
I'm trying file download with golang.
I'm downloading file it's okay. After I'm using cheggaaa's progressbar library. But I can't dynamic.
How can I do dynamic progressbar?
My code below:
package main
import (
"flag"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"github.com/cheggaaa/pb"
"time"
)
/*
usage = usage text
version = current number
help use Sprintf
*cliUrl from cmd
*cliVersion from cmd
*cliHelp * from cmd
*/
var (
usage = "Usage: ./gofret -url=http://some/do.zip"
version = "Version: 0.1"
help = fmt.Sprintf("\n\n %s\n\n\n %s", usage, version)
cliUrl *string
cliVersion *bool
cliHelp *bool
)
func init() {
/*
if *cliUrl != "" {
fmt.Println(*cliUrl)
}
./gofret -url=http://somesite.com/somefile.zip
./gofret -url=https://github.com/aligoren/syspy/archive/master.zip
*/
cliUrl = flag.String("url", "", usage)
/*
else if *cliVersion{
fmt.Println(flag.Lookup("version").Usage)
}
./gofret -version
*/
cliVersion = flag.Bool("version", false, version)
/*
if *cliHelp {
fmt.Println(flag.Lookup("help").Usage)
}
./gofret -help
*/
cliHelp = flag.Bool("help", false, help)
}
func main() {
/*
Parse all flags
*/
flag.Parse()
if *cliUrl != "" {
fmt.Println("Downloading file")
/* parse url from *cliUrl */
fileUrl, err := url.Parse(*cliUrl)
if err != nil {
panic(err)
}
/* get path from *cliUrl */
filePath := fileUrl.Path
/*
seperate file.
http://+site.com/+(file.zip)
*/
segments := strings.Split(filePath, "/")
/*
file.zip filename lenth -1
*/
fileName := segments[len(segments)-1]
/*
Create new file.
Filename from fileName variable
*/
file, err := os.Create(fileName)
if err != nil {
fmt.Println(err)
panic(err)
}
defer file.Close()
/*
check status and CheckRedirect
*/
checkStatus := http.Client{
CheckRedirect: func(r *http.Request, via []*http.Request) error {
r.URL.Opaque = r.URL.Path
return nil
},
}
/*
Get Response: 200 OK?
*/
response, err := checkStatus.Get(*cliUrl)
if err != nil {
fmt.Println(err)
panic(err)
}
defer response.Body.Close()
fmt.Println(response.Status) // Example: 200 OK
/*
fileSize example: 12572 bytes
*/
fileSize, err := io.Copy(file, response.Body)
/*
progressbar worked after download :(
*/
var countSize int = int(fileSize/1000)
count := countSize
bar := pb.StartNew(count)
for i := 0; i < count; i++ {
bar.Increment()
time.Sleep(time.Millisecond)
}
bar.FinishPrint("The End!")
if err != nil {
panic(err)
}
fmt.Printf("%s with %v bytes downloaded", fileName, count)
} else if *cliVersion {
/*
lookup version flag's usage text
*/
fmt.Println(flag.Lookup("version").Usage)
} else if *cliHelp {
/*
lookup help flag's usage text
*/
fmt.Println(flag.Lookup("help").Usage)
} else {
/*
using help's usage text for handling other status
*/
fmt.Println(flag.Lookup("help").Usage)
}
}
While the my program is running:
Downloading file
200 OK
After download working progressbar:
6612 / 6612 [=====================================================] 100.00 % 7s
The End!
master.zip with 6612 bytes downloaded
My progressbar code below:
/*
progressbar worked after download :(
*/
var countSize int = int(fileSize/1000)
count := countSize
bar := pb.StartNew(count)
for i := 0; i < count; i++ {
bar.Increment()
time.Sleep(time.Millisecond)
}
bar.FinishPrint("The End!")
How can I solve progressbar problem?
More goroutines is not needed. Just read from bar
// start new bar
bar := pb.New(fileSize).SetUnits(pb.U_BYTES)
bar.Start()
// create proxy reader
rd := bar.NewProxyReader(response.Body)
// and copy from reader
io.Copy(file, rd)
I wrote the below stuff, and it's right in the general case, unrelated to progressbar, but this library is exactly designed to handle this problem, has specialized support for it, and gives an explicit example of downloading a file.
You need to run the download in parallel with updating the progressbar, currently you're downloading the whole file and then updating the progressbar.
This is a little sloppy, but should get you going in the right direction:
First, grab the expected filesize:
filesize := response.ContentLength
Then start your download in a goroutine:
go func() {
n, err := io.Copy(file, response.Body)
if n != filesize {
log.Fatal("Truncated")
}
if err != nil {
log.Fatalf("Error: %v", err)
}
}()
Then update your progressbar as it goes along:
countSize := int(filesize / 1000)
bar := pb.StartNew(countSize)
var fi os.FileInfo
for fi == nil || fi.Size() < filesize {
fi, _ = file.Stat()
bar.Set(int(fi.Size() / 1000))
time.Sleep(time.Millisecond)
}
bar.FinishPrint("The End!")
Like I say, this is a little sloppy; you probably want to scale the bar better depending on the size of the file, and the log.Fatal calls are ugly. But it handles the core of the issue.
Alternately, you can do this without a goroutine just by writing your own version of io.Copy. Read a block from response.Body, update the progress bar, and then write a block to file. That's arguably better because you can avoid the sleep call.
Actually you can implement progress bar by yourself with below piece of code.
func (bar *Bar) Play(cur int64) {
bar.cur = cur
last := bar.percent
bar.percent = bar.getPercent()
if bar.percent != last && bar.percent%2 == 0 {
bar.rate += bar.graph
}
fmt.Printf("\r[%-50s]%3d%% %8d/%d", bar.rate, bar.percent, bar.cur, bar.total)
}
The key here is to use escape \r which will replace the current progress with updated one in place which creates a dynamic effect.
A detailed explanation can be found at here.