Go programs won't run after MacOS Catalina upgrade - macos

This might be a weird question, but some programs written in Go won't run on my MacBook Pro after MacOS Catalina upgrade.
However a basic "Hello World" program runs, but then I am using the writing a program using net/http package I get the following error:
Note: The programs are correct, they run fine inside a Docker container
Output (IntelliJ):
# runtime/cgo
In file included from gcc_darwin_amd64.c:6:
/usr/local/include/pthread.h:331:6: error: macro expansion producing 'defined' has undefined behavior [-Werror,-Wexpansion-to-defined]
/usr/local/include/pthread.h:200:2: note: expanded from macro '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
/usr/local/include/pthread.h:331:6: error: macro expansion producing 'defined' has undefined behavior [-Werror,-Wexpansion-to-defined]
/usr/local/include/pthread.h:200:34: note: expanded from macro '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
/usr/local/include/pthread.h:540:6: error: macro expansion producing 'defined' has undefined behavior [-Werror,-Wexpansion-to-defined]
/usr/local/include/pthread.h:200:2: note: expanded from macro '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
/usr/local/include/pthread.h:540:6: error: macro expansion producing 'defined' has undefined behavior [-Werror,-Wexpansion-to-defined]
/usr/local/include/pthread.h:200:34: note: expanded from macro '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
Things that I have already tried:
Reinstall go
Reinstall xcode
Check if GOPATH & GOROOT are properly set
Used:
MacOS version - Catalina 10.15.6
Go version - go1.15.2 darwin/amd64

Type xcode-select -print-path in terminal window and check your installed directory. In my case the output was: /Applications/Xcode.app/Contents/Developer
If it's the same case then using the following command on your terminal can fix things:
sudo xcode-select --switch /Library/Developer/CommandLineTools
Another option is to use CGO_CPPFLAGS as mentioned in this issue, but it would work within the session:
export CGO_CPPFLAGS="-Wno-error -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-builtin-requires-header"

Sadly, none of the tryouts worked.
I have been using IntelliJ IDEA Ultimate for sometime now.. so I tried to setup Go SDK, GOROOT and GOPATH using the IDE (just a thought came to my mind).
I uninstalled/deleted everything related to Go on my MacBook.
Then I installed the Go plugin on IntelliJ IDEA and as expected it prompted that Go SDK is not available, neither GOROOT and GOPATH are set.
I followed the instructions and IntelliJ IDEA took care of the rest!
It downloaded and installed Go SDK, asked me to select GOPATH, it indexed stuff and now every thing is working like a charm!

This is a bug in the <pthread.h> that ships with macOS tools:
This macro is even bogus for more than one reason! The definition in <pthread.h> is this:
#define _PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT \
defined(SWIFT_CLASS_EXTRA) && (!defined(SWIFT_SDK_OVERLAY_PTHREAD_EPOCH) || (SWIFT_SDK_OVERLAY_PTHREAD_EPOCH < 1))
The macro expansion is not properly parenthesized. The macro is used this way in the same file:
#if !_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT
Which expands to:
#if !defined(SWIFT_CLASS_EXTRA) && (!defined(SWIFT_SDK_OVERLAY_PTHREAD_EPOCH) || (SWIFT_SDK_OVERLAY_PTHREAD_EPOCH < 1))
The ! only applies to the first test defined(SWIFT_CLASS_EXTRA) not to the whole boolean expression.
The compiler does not detect this problem, it just complains about the defined preprocessor operator coming from a macro expansion, which has undefined behavior as specified in the C Standard:
6.10.1 Conditional inclusion
[...]
Constraints
1 The expression that controls conditional inclusion shall be an integer constant expression except that: identifiers (including those lexically identical to keywords) are interpreted as described below; and it may contain unary operator expressions of the form
defined identifier
or
defined ( identifier )
which evaluate to 1 if the identifier is currently defined as a macro name (that is, if it is predefined or if it has been the subject of a #define preprocessing directive without an intervening #undef directive with the same subject identifier), 0 if it is not.
2 Each preprocessing token that remains (in the list of preprocessing tokens that will become the controlling expression) after all macro replacements have occurred shall be in the lexical form of a token (6.4).
Semantics
3 Preprocessing directives of the forms
# if constant-expression new-line groupopt
# elif constant-expression new-line groupopt
check whether the controlling constant expression evaluates to nonzero.
4 Prior to evaluation, macro invocations in the list of preprocessing tokens that will become the controlling constant expression are replaced (except for those macro names modified by the defined unary operator), just as in normal text. If the token defined is generated as a result of this replacement process or use of the defined unary operator does not match one of the two specified forms prior to macro replacement, the behavior is undefined. After all replacements due to macro expansion and the defined unary operator have been performed, all remaining identifiers (including those lexically identical to keywords) are replaced with the pp-number 0, and then each preprocessing token is converted into a token. The resulting tokens compose the controlling constant expression which is evaluated according to the rules of 6.6. For the purposes of this token conversion and evaluation, all signed integer types and all unsigned integer types act as if they have the same representation as, respectively, the types intmax_t and uintmax_t defined in the header <stdint.h>. This includes interpreting character constants, which may involve converting escape sequences into execution character set members. Whether the numeric value for these character constants matches the value obtained when an identical character constant occurs in an expression (other than within a #if or #elif directive) is implementation-defined. Also, whether a single-character character constant may have a negative value is implementation-defined.
This relevant phrase is If the token defined is generated as a result of this replacement process or use of the defined unary operator does not match one of the two specified forms prior to macro replacement, the behavior is undefined.
It is unclear if generated just means produced as part of the macro expansion or more specifically produced by token pasting. clang seems to consider that any defined token produced coming from a macro expansion has undefined behavior in a #if controlling expression.
For both reasons, this seems to be a bug in <pthread.h> for Apple to fix.

Related

Ghostscript -d vs -s command line parameters

I recently upgraded Ghostscript from 9.10 to 9.53.3 (also tried 9.50 first). Scripts that have run for years started failing with
Unrecoverable error: rangecheck in .putdeviceprops
After some research and trial and error testing, it seems that Ghostscript changed some of the command line switches from "-s" to "-d". for example:
-sGrayImageResolution=600 now errors but -dGrayImageResolution=600 does not.
Some switches appear to accept either form. For example:
-sColorImageResolution=600 and -dColorImageResolution=600 both work.
(Note: When I say "work" I mean they do not throw the error.)
I have 2 questions
Where can I find a complete list of Ghostscript command line parameters? The Ghostscript documents seem to be incomplete.
What is the difference between -s and -d for a switch? (this is really just a curiosity question)
Thanks
The ColorImageResolution and GrayImageResolution are PostScript distiller parameters found in the VectorDevices.htm#PDFWRITE so are used with setdistillerparams and currentdistillerparams in the PostScript code for -sDEVICE=pdfwrite. Also see TN 5151 Acrobat Distiller Parameters. For example:
<< /MonoImageResolution 72 >> setdistillerparams
EDIT: Here are some more: distillerparams
Some settings can be defined in the system dictionary without error except are never used for anything. Be sure to check all of the documentation. There are some settings not listed in the documentation that can be found by searching in the ghostscript Resource/Init files especially for advanced users. Some of these are unique to ghostscript and beyond normal PostScript.
This is from the ghostscript Use.htm#Options :
-Dname
-dname
Define a name in systemdict with value=true.
-Dname=token
-dname=token
Define a name in systemdict with the given value. The value must be a valid PostScript token (as defined by the token operator). If the
token is a non-literal name, it must be true, false, or null. It is
recommeded that this is used only for simple values -- use -c (above)
for complex values such as procedures, arrays or dictionaries.
Note that these values are defined before other names in systemdict, so any name that conflicts with one usually in systemdict
will be replaced by the normal definition during the interpreter
initialization.
-Sname=string
-sname=string
Define a name in systemdict with a given string as value. This is different from -d. For example, -dXYZ=35 on the command line is
equivalent to the program fragment
/XYZ 35 def
whereas -sXYZ=35 is equivalent to
/XYZ (35) def

Set Bash Variable to another causes operand error

I've looked through several answers on stack overflow relating to setting a bash variable resulting in a "syntax error: operand expected" and tried some suggestions without success. I'm new to bash and Linux in general so I hope someone can help.
Basically my coworker wants me to run the following in terminal:
export TOPDIR=/home/user/folder/subfolder
export TOP=$TOPDIR
However I get an error on the second line:
bash: export: /home/user/folder/subfolder: syntax error: operand expected (error token is "/home/user/folder/subfolder")
I have tried changing the line to some variations such as
export TOP=${TOPDIR}
and
export TOP="${TOPDIR}"
without success.
Any suggestions or help on what I am doing wrong would be appreciated.
NOTE: I tried setting TOP to the file location directly without success. If I echo $TOP I get 0 returned.
echo $BASH_VERSION = 4.2.46(2) - release
The problem is that TOP has been declared as an integer (using declare -i TOP or equivalent).
From the declare documentation:
-i
The variable is to be treated as an integer; arithmetic evaluation (see Shell Arithmetic) is performed when the variable is assigned a value.
That's why
export TOP=$TOPDIR
tries to evaluate /home/user/folder/subfolder as an arithmetic expression and fails (because it's a syntax error).
Possible ways to proceed:
Find out where and why TOP was declared as an integer1 and (if it was not intended) remove it.
Use a different variable name.
Use declare +i TOP to remove the integer attribute from TOP.
1 Possible culprit: The gluster bash completion script declares TOP as an integer and leaks it into the global environment. There's a patch that fixes the problem (or rather moves it; it just renames TOP to GLUSTER_TOP).

Why can't environment variables with dashes be accessed in bash 4.1.2?

On a CentOS 5 host (with bash 3.2.32), we use Ruby (1.8.7) to
ENV['AWS_foo-bar_ACCESS_KEY'] = xxxxx
Then, using bash, we run a shell script that does:
BUCKET_NAME=$1
AWS_ACCESS_KEY_ID_VAR="AWS_${BUCKET_NAME}_ACCESS_KEY_ID"
AWS_ACCESS_KEY_ID="${!AWS_ACCESS_KEY_ID_VAR}"
export AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
This works fine on CentOS 5.
However, on CentOS 6 (with bash 4.1.2), we get the error
-bash: export: `AWS_foo-bar_ACCESS_KEY_ID=xxxxx': not a valid identifier
It is our understanding that this fails because - is not allowed in the variable name. But why does this work on bash 3.2 and not bash 4.1?
The "why" is almost irrelevant: The POSIX standard makes it very clear that export is only required to support arguments which are valid names, and anything with a dash is not a valid name. Thus, no POSIX shell is required to support exporting or expanding variable names with dashes, via indirect expansion or otherwise.
It's worth noting that ShellShock -- a major security bug caused by sloppy handling of environment contents -- is fixed in the bash 4.1 present in the current CentOS 6 updates repo; increased rigor in an area which spawned security bugs should be no surprise.
The remainder of this answer will focus on demonstrating that the new behavior of bash 4.1 is explicitly allowed, or even required, by POSIX -- and thus that the prior behavior was an undefined implementation artifact.
To quote POSIX on environment variables:
These strings have the form name=value; names shall not contain the character '='. For values to be portable across systems conforming to IEEE Std 1003.1-2001, the value shall be composed of characters from the portable character set (except NUL and as indicated below). There is no meaning associated with the order of strings in the environment. If more than one string in a process' environment has the same name, the consequences are undefined.
Environment variable names used by the utilities in the Shell and Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_' (underscore) from the characters defined in Portable Character Set and do not begin with a digit. Other characters may be permitted by an implementation; applications shall tolerate the presence of such names. Uppercase and lowercase letters shall retain their unique identities and shall not be folded together. The name space of environment variable names containing lowercase letters is reserved for applications. Applications can define any environment variables with names from this name space without modifying the behavior of the standard utilities.
Note: Other applications may have difficulty dealing with environment variable names that start with a digit. For this reason, use of such names is not recommended anywhere.
Thus:
Tools (including the shell) are required to fully support environment variable names with uppercase and lowercase letters, digits (except in the first position), and the underscore.
Tools (including the shell) may modify their behavior based on environment variables with names that comply with the above and additionally do not contain lowercase letters.
Tools (including the shell) should tolerate other names -- meaning they shouldn't crash or misbehave in their presence -- but are not required to support them.
Finally, shells are explicitly allowed to discard environment variable names which are not also shell variable names. From the relevant standard:
It is unspecified whether environment variables that were passed to the shell when it was invoked, but were not used to initialize shell variables (see Shell Variables) because they had invalid names, are included in the environment passed to execl() and (if execl() fails as described above) to the new shell.
Moreover, what defines a valid shell name is well-defined:
Name - In the shell command language, a word consisting solely of underscores, digits, and alphabetics from the portable character set. The first character of a name is not a digit.
Notably, only underscores (not dashes) are considered part of a valid name in a POSIX-compliant shell.
...and the POSIX specification for export explicitly uses the word "name" (which it defined in the text quoted above), and describes it as applying to "variables" (shell variables, the restrictions on names for which are also subject to restrictions quoted elsewhere in this document):
The shell shall give the export attribute to the variables corresponding to the specified names, which shall cause them to be in the environment of subsequently executed commands. If the name of a variable is followed by = word, then the value of that variable shall be set to word.
All the above being said -- if your operating system provides a /proc/self/environ which represents the state of your enviroment variables at process startup (before a shell has, as it's allowed to do, potentially discarded any variables which don't have valid names in shell), you can extract content with invalid names like so:
# using a lower-case name where possible is in line with POSIX guidelines, see above
aws_access_key_id_var="AWS_${BUCKET_NAME}_ACCESS_KEY_ID"
while IFS= read -r -d '' var; do
[[ $var = "$aws_access_key_id_var"=* ]] || continue
val=${var#"${aws_access_key_id_var}="}
break
done </proc/self/environ
echo "Extracted value: $val"

Coq --- Arguments directive

I am reading Software foundations book and I came across a command that declares parameters
as implicit:
Arguments nil {X}.
where, for example:
Inductive list (X:Type) : Type :=
| nil : list X
| cons : X -> list X -> list X.
However, whenever I try to execute such commands I get the following message:
Error: No focused proof (No proof-editing in progress).
The same message appears even if I try to compile scripts that come with the book. What could be the problem?
I am using Coq version 8.3pl4 and CoqIDE editor.
I just tried it on my (somewhat old) Coq 8.4 and I don't have any problem with the implicit declaration.
However if I write Argument instead of Arguments (notice the lack of "s"), I get
Error: Unknown command of the non proof-editing mode.
Did you correctly spelled it ?
EDIT: sorry, I miss-read your version. It seems that the Arguments command has been added post 8.4 (it does not appear here but appears here. I advise you update your Coq version if possible, or restrict to using 8.3 Implicit related commands (wild guess: Implicit Arguments foo.)

How do leading key=value pairs on shell commands work? Why are they not an error?

(my_virtualenv)my_pc:~/path$ ASDF='asdf' python
...
>>> import os
>>> os.environ['ASDF']
'asdf'
So how does this work? Why doesn't the interpreter look for the ASDF command, and report an error if it doesn't find it?
Because leading var=value pairs are recognized as command-specific environment variable names as part of the standard for shells compatible with POSIX sh.
From 2.10.2 ("Shell Grammar Rules") of the Shell Command Language specification:
7. [Assignment preceding command name]
7.a. [When the first word]
If the TOKEN does not contain the character '=', rule 1 is applied. Otherwise, 7b shall be applied.
7.b. [Not the first word]
If the TOKEN contains the equal sign character:
If it begins with '=', the token WORD shall be returned.
If all the characters preceding '=' form a valid name (see the Base Definitions volume of IEEE Std 1003.1-2001, Section 3.230, Name), the token ASSIGNMENT_WORD shall be returned. (Quoted characters cannot participate in forming a valid name.)
Otherwise, it is unspecified whether it is ASSIGNMENT_WORD or WORD that is returned.
Assignment to the NAME shall occur as specified in Simple Commands.
From 2.9.1 ("Simple Commands") of the Shell Command Language specification:
If no command name results, variable assignments shall affect the current execution environment. Otherwise, the variable assignments shall be exported for the execution environment of the command and shall not affect the current execution environment (except for special built-ins). [...]
Emphasis added.

Resources