why do I get a locale error even though it is set? - utf-8

When I run bitbake, I'm getting the following:
$ bitbake core-image-base
Please use a locale setting which supports utf-8.
Python can't change the filesystem locale after loading so we need a utf-8 when python starts or things won't work.
even though my locale is set to en_US.UTF-8, why is this?
$ echo $LC_ALL
en_US.UTF-8
For additional background information, please also see https://unix.stackexchange.com/questions/626916/how-to-set-locale-correctly-manually/626919
UPDATE:
$ locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8
My ~/.bashrc looks like:
$ cat ~/.bashrc
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
and when opening a new shell I get:
$ bash
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

If the shell existed before you added the locale, then you need to open a new one (either running bash as a child, opening a new terminal, doing a new ssh,...)
Then this should work.
$ export LC_ALL="en_US.UTF-8"
$ bitbake core-image-base
The export might not be needed, that depends on the default for your system.

Making the "comment crowned by success" an answer:
sudo locale-gen en_US en_US.UTF-8
sudo dpkg-reconfigure locales

Related

Not able to set lang in zsh macos

I need my lang to be set to en_US.UTF-8
I followed a lot of threads and resources on stackoverflow as well as other resources but none of them worked for me.
Here is the issue :
I am using 10.15.1 MacOs Catalina and have set Zsh as my default shell.
Here are the profile files :
~/.zshrc
export PATH="/Applications/flutter/bin:$PATH"
export PATH="$PATH":"/Applications/flutter/.pub-cache/bin"
export LC_ALL="en-US.UTF-8"
export PATH="/usr/local/opt/ruby/bin:$PATH"
export LDFLAGS="-L/usr/local/opt/ruby/lib"
export CPPFLAGS="-I/usr/local/opt/ruby/include"
export PKG_CONFIG_PATH="/usr/local/opt/ruby/lib/pkgconfig"
~/.profile
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
~/.bash_profile
# Setting PATH for Python 3.7
# The original version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.7/bin:${PATH}"
export PATH
# added by Anaconda3 5.2.0 installer
export PATH="/anaconda3/bin:$PATH"
#For flutter
export PATH="/Applications/flutter/bin:$PATH"
export PATH="$PATH":"/Applications/flutter/.pub-cache/bin"
export LC_ALL="en-US.UTF-8"
# Setting PATH for Python 3.7
# The original version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/3.7/bin:${PATH}"
export PATH
#setting for ruby
export PATH="/usr/local/opt/ruby/bin:$PATH"
Here is the output of locale :
LANG=""
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL="C"
I need everything to be set to en_US.UTF-8, how do i do it ?
First, only .zshrc is relevant; the other two files are not used by zsh at all.
I don't fully understand how locales work, and how they work in macOS specifically, but the solution would appear to be to set LANG in your .zshrc file.
export LANG="en_US.UTF-8"
That said, you should put environment variables in .zprofile, rather than .zshrc, because they only need to be set once. Any non-login shell that would only execute .zshrc will inherit environment variables from the parent; they don't need to be set again. (It's less important with zsh than with bash, which only sources .bash_profile for a login shell, not it and .bashrc. zsh will source both files automatically)

File missing upon launching EC2 using Amazon Linux AMI

I've just launched an t2.micro free tier EC2 instance and SSH into it from my local machine. And I was welcomed by an error message as below:
-bash: warning: setlocale: LC_CTYPE: cannot change locale (UTF-8): No such file or directory
This instance was launch using the console and every option has been by default and nothing special being configured. What could be the reason for it?
Below is the Locale of this instance:
[root#ip-xxx .aws]# locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LC_CTYPE=UTF-8
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
in my case it was required to add following lines to /etc/environment
LANG=en_US.utf-8
LC_ALL=en_US.utf-8
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
Also for later use add the same two commands to ~/.bash_profile
Based on #ardit comment

Changing Locale in Docker Stops Many Commands From Executing?

How come when I change my locale in a Dockerfile using this...:
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
...so that I can achieve a change in locale from this...
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
..to this..
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8
Then I get issues, such as this with no commands executing:
root#820760edeb77:/# irb
bash: irb: command not found
But if I take those changes to the locale, and leave them as they are after a rebuilt container, then everything works as expected??:
# ENV LANG en_US.UTF-8
# ENV LANGUAGE en_US:en
# ENV LC_ALL en_US.UTF-8
root#820760edeb77:/# irb
2.3.1 :001 >
I'm not too sure what would be causing this issue, whereby the changed locale inhibits commands from working, but suspect this could just be a side effect of such a change in Locale within the Docker container, and possibly not the real issue?
So I just figured it out, turns out I was using the wrong locale type for Docker, which is related to this issue here.
A small, but critical distinction C vs en_US:
WRONG
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
RIGHT
ENV LANG C.UTF-8
ENV LANGUAGE C.UTF-8
ENV LC_ALL C.UTF-8
Can anyone tell me why Docker uses C as a locale, vs en_US, or any other?
Now Ruby/irb is successfully working (albeit, with 4 hours of my life lost.....)

Why doesn't Encoding.default_external respect LANG?

It's my understanding that Ruby's Encoding.default_external is given a default value based on the environment variables LC_ALL and LANG, giving precedence to the former. I've run into several bugs where the default external encoding somehow ends up set to ASCII even though the environment variables are set to UTF-8.
For example:
$ irb
irb(main):001:0> Encoding.default_external
=> #<Encoding:US-ASCII>
irb(main):002:0> ENV['LC_ALL']
=> nil
irb(main):003:0> ENV['LANG']
=> "en_US.UTF-8"
In the environments where this has happened, I've also grepped through all the gems being loaded for any code manually setting the default external encoding, but haven't found anything. How is what I'm seeing above possible? I'm using Ruby 2.2 above, but I've seen this happen on all Ruby 2.x versions.
I figured it out. Not only does the LANG environment variable need to be set, but the locale it species must have been generated for the OS. On a stock Linux image, the default locale may be something that is not UTF-8. In my particular case, I'm using Debian 7.7 and the default locale is "POSIX". I was able to set the default locale by installing the locales package and following the interactive prompts to generate the en_US.UTF-8 locale:
$ apt-get -y install locales
If the locales package is already installed, you can just reconfigure it instead:
$ dpkg-reconfigure locales
Now setting LANG will change the current system locale, and Ruby's Encoding.default_external will be set properly:
$ export LANG=en_US.UTF-8
$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
$ irb
irb(main):001:0> Encoding.default_external
=> #<Encoding:UTF-8>
For an example of how to automate the generation and configuration of the default locale instead of doing it interactively, take a look at this Docker image.

how to show different language characters in git log?

Here is what I am seeing now in my git log:
commit d0ed97d243eaf905b03244a4d1ba6af26213e4ad
Author: some guy <someone#somesite.com>
Date: Mon Jun 3 11:22:06 2013 +0900
<E3><83><98><E3><83><83><E3><83><80><E3><81><AE><E3><83><A6><E3><83><BC><E3><82><B6><E3><82><A2><E3><82><A4><E3><82><B3><E3><83><B3><E8><A1><A8><E7><A4><BA>
the commit is made by some Japanese characters.
I've already changed my LANG env to ja_JP.UTF-8, en_US.UTF-8, ja_JP.eucJP:
$ export LANG=ja_JP.UTF-8
but nothing worked.
Hope someone knows and give me some points.
Thanks.
UPDATE (solution 1):
Finally, I managed to show those Japanese characters by commenting out the line SendEnv LANG LC_* in /etc/ssh_config file in the local environment(which is MacOS).
But I don't think I solved this issue. I think I just cut off the way of sending the settings of LANG to the server side(CentOS). So basically, the LANG settings in the local env is still messing up because when I tried $ locale in the local env(mac), it shows the following settings:
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=
Which is obviously a inappropriate setting.
I am still undering goggling...
UPDATE (solution 2):
I finally find the solution to the "locale" issue,
here is the link I find:How to fix locale issues in Mac OSX terminal
Thanks for all the help from all the people here.
Make sure that your system supports UTF-8 properly. Most Ubuntu and Fedora boxes do. If you are using terminal or ssh session, make sure that your ssh client is configured to use UTF-8 all the way. For example, if using putty, be sure to configure it to use UTF-8 encoding in preferences - it is not a default setting.
I have relatively old git version 1.7.7.6 on this Fedora 16 box, but it seems to work just fine, as long as I set my locale properly:
$ export LANG=en_US.UTF-8
$ git log
commit 40b8cb9fc58775826a511c5338370f7fd1454481
Author: some guy <someone#somesite.com>
Date: Wed Jun 5 22:57:10 2013 -0700
welcome
привет
ヘッダのユー
It still works the same if I set locale to Japanese:
$ export LANG=ja_JP.UTF-8
$ git log
commit 40b8cb9fc58775826a511c5338370f7fd1454481
Author: some guy <someone#somesite.com>
Date: Wed Jun 5 22:57:10 2013 -0700
welcome
привет
ヘッダのユー
But it breaks if I set locale to Posix (C):
$ export LANG=C
$ git log
commit 40b8cb9fc58775826a511c5338370f7fd1454481
Author: some guy <someone#somesite.com>
Date: Wed Jun 5 22:57:10 2013 -0700
welcome
<D0><BF><D1><80><D0><B8><D0><B2><D0><B5><D1><82>
<E3><83><98><E3><83><83><E3><83><80><E3><81><AE><E3><83><A6><E3><83><BC>
For reference, this is what working locale should look like:
$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Resources