python lambda not able to import user defined modules - aws-lambda

I have a python lambda which turns out to be a bit long so we decided to split it into modules. and now when i try to import the module in the lambda_handler its seems to be giving following error
Unable to import module 'defghi': attempted relative import with no known parent package
abc.py which has got lambda_handler in it, is trying to import the defghi.py methods as follows
from defghi import some_method_1, some_method_2
tried this as well
from .defghi import some_method_1, some_method_2
both the files are in the same directory
any sort of help would be appreciated, Thanks in adavance

Finally got that working it was the build scripts which where causing the issue in my project.
The build script which created the zip file where expecting only one .py file in the directory which was a problem. So the zip created never had other files which then gave error cannot import.
So to answer the question its perfectly fine to split the big lambdas into modules and keep them a bit readable and import into the main lambda_handler the required modules.

Related

Huggingface AutoTokenizer cannot be referenced when importing Transformers

I am trying to import AutoTokenizer and AutoModelWithLMHead, but I am getting the following error:
ImportError: cannot import name 'AutoTokenizer' from partially initialized module 'transformers' (most likely due to a circular import)
First, I install transformers: pip install transformers then implemented the following code:
from transformers import AutoTokenizer, AutoModelWithLMHead
tokenizer = AutoTokenizer.from_pretrained("t5-base")
model = AutoModelWithLMHead.from_pretrained("t5-base")
For anyone who comes across a problem around circular import, this could be due to the naming convention of your .py file. Changing my file name solved the issue as there might be a file in my Python lib folder with similar naming conventions.

Conflict in repo name and module name

I created the github.com project qjson/qjson-go that contains the package name qjson that you can see here. I named the github project this way because github.com/qjson/ contains other projects for different languages (e.g. qjson-c).
Unfortunately, I get the following error when I try to import the project as github.com/qjson/qjson-go:
$ go mod tidy
go: finding module for package github.com/qjson/qjson-go
go: downloading github.com/qjson/qjson-go v0.0.0-20210128102242-170c47e2db46
github.com/xxx/xxx imports
github.com/qjson/qjson-go: module github.com/qjson/qjson-go#latest found (v0.0.0-20210128102242-170c47e2db46), but does not contain package github.com/qjson/qjson-go
I’m apparently doing it wrong. I understand that due to the import statement we are then expected to use gjson-go as package identifier.
What must I do so that the git project can be named qjson-go and the package qjson ?
I assume that one solution is to create a sub-directory named qjson inside qjson-go and move all package files in it. The user would then import "github.com/qson/qson-go/qjson". Is that correct ? Is there another solution avoiding the stutter ?
This program works as expected:
package main
import (
"fmt"
"github.com/qjson/qjson-go/qjson"
)
func main() {
fmt.Println(qjson.ErrDivisionByZero)
}
The issue is that you are using this file structure:
qjson/engine.go
qjson/errors.go
When you should just be putting them at the top level, like this:
engine.go
errors.go
So you can either fix the directory, and tag a new version, or just leave the
files as is, and change your imports to match what I have above.

Run tag-dependent Python script on Sphinx build

I need to run a simple Python script to copy a set of files from one directory to another during the build-phase of my Sphinx documentation.
Copying function:
location: source/_plugins/copy_firmware_files.py
import json, os, sys
from pathlib import Path
import shutil
def copy_firmware_files(device):
# copy firmware files
I'm currently importing this module into my conf.py as the Configuration File contains the device name, which would make it a simple way to execute the code. I'm currently doing this as below:
Configuration File (conf.py)
location: source/conf.py
sys.path.append(os.path.abspath("_plugins"))
from copy_firmware_files import *
# initialize files depending on build
copy_firmware_files(device_name)
The above works as intended, i.e. the relevant files are copied to their respective folders before the build. However, I'm unsure if it's the "proper" way to do so. Is there a more correct way of achieving the same result?
There are several ways, but it sounds like two are most appropriate. They are static files and could be treated as such by Sphinx.
html_extra_path is one option.
If you want to present links to download the files, you can use the download directive.
See :download:`this example script <../example.py>`.

Code in directory expects import

I have a package which is forked from a repository myproject. Inside the project, I want to use some functions from sha3 package, however, I need to first add a go file to sha3 package which contains some extra functionalities. I want to include this custom sha3 package inside my project. I copied and pasted the sha3 directory into myproject directory, and inside my go codes, I imported sha3 package as:
import . "github.com/myproject/sha3". Now, when I try to build myproject package, I am getting:
code in directory /src/github.com/myproject/sha3 expects import "golang.org/x/crypto/sha3". I cannot understand what the problem is. I checked all the go files inside sha3 directory and none of them requires any import!
line number 66 sha3/docs.go has the import comment.
import "golang.org/x/crypto/sha3"
You can get rid of the build error by removing that.

Building package structure with child-/sub-packages

I'm trying to make a simple calculator in Go. I'm designing it in such a way that I can build a command-line interface first and easily swap in a GUI interface. The project location is $GOPATH/src/gocalc (all paths hereafter are relative to the project location). The command-line interface logic is stored in a file gocalc.go. The calculator logic is stored in files calcfns/calcfns.go and operations/operations.go. All files have package names identical to their filename (sans extension) except the main program, gocalc.go, which is in the package main
calcfns.go imports operations.go via import "gocalc/operations"; gocalc.go imports calcfns.go via import "gocalc/calcfns"
To summarize:
$GOPATH/src/gocalc/
gocalc.go
package main
import "gocalc/calcfns"
calcfns/
calcfns.go
package calcfns
import "gocalc/operations"
operations/
operations.go
package operations
When I try to go build operations (from the project dir), I get the response: can't load package: package operations: import "operations": cannot find package
When I try go build gocalc/operations, I get can't load package: package gocalc/operations: import "gocalc/operations": cannot find package
When I try go build operations/operations.go, it compiles fine
When I try to go build calcfns or go build gocalc/calcfns, I get can't load package... messages, similar to those in operations; however, when I try to build calcfns/calcfns.go it chokes on the import statement: import "gocalc/operations": cannot find package
Finally, when I try go build . from the project dir, it chokes similar to the previous paragraph: import "gocalc/calcfns": cannot find package
How should I structure my child packages and/or import statements in such a way that go build won't fail?
Stupidly, I forgot to export my GOPATH variable, so go env displayed "" for GOPATH. (thanks to jnml for suggesting to print go env; +1).
After doing this (and moving the main program to its own folder {project-dir}/gocalc/gocalc.go), I could build/install the program via go install gocalc/gocalc.
Moral of the story, make sure you type export GOPATH=... instead of just GOPATH=... when setting your $GOPATH environment variable
Please try to also add output of $ go env to provide more clues. Otherwise both the directories structure and (the shown) import statements looks OK.
However the sentence
When I try to go build operations (from the project dir), I get the response: can't load package: package operations: import "operations": cannot find package
sounds strange. It seems to suggest you have
package operations
import "operations"
in 'operations.go', which would be the culprit then...?
Very easy:
Lets say I have a project/app named: golang-playground
Put your root dir under GOPATH/src/, in my case GOPATH=~/go/src (run command go env to get your GOPATH). so complete path for my app is ~/go/src/golang-playground
Lets say you want to use function Index() inside of file: router.go from my main.go file (which of course is on root dir). so in main.go:
import (
...
"golang-playground/router"
)
func main() {
foo.Bar("/", router.Index) // Notice caps means its public outside of file
}

Resources