I have a text file that has some ® (Registered Trade Mark) symbols in it. The file is in UTF-8.
I'm trying to import this file and populate a MySQL database using Rails 3. The DB appears to be setup fine to take UTF-8
+-------------+--------------+-----------------+------+-----+---------+----------------+---------------------------------+---------+
| Field | Type | Collation | Null | Key | Default | Extra | Privileges | Comment |
+-------------+--------------+-----------------+------+-----+---------+----------------+---------------------------------+---------+
| id | int(11) | NULL | NO | PRI | NULL | auto_increment | select,insert,update,references | |
| user_id | int(11) | NULL | YES | MUL | NULL | | select,insert,update,references | |
| title | varchar(255) | utf8_general_ci | YES | | NULL | | select,insert,update,references | |
| translation | text | utf8_general_ci | YES | | NULL | | select,insert,update,references | |
| created_at | datetime | NULL | NO | | NULL | | select,insert,update,references | |
| updated_at | datetime | NULL | NO | | NULL | | select,insert,update,references | |
+-------------+--------------+-----------------+------+-----+---------+----------------+---------------------------------+---------+
Yet, when I try to do this:
trans_file = params[:descriptions] #coming from file_field_tag
trans = trans_file.read.split("\r\n")
trans.each do |tran|
ttl = ''
desc = ''
tran.split(']=').each do |title|
if title =~ /\[/ #it's the title
ttl = title.sub('[','')
else
desc = title.gsub('FFF', "\r\n")
end
end
#current_user.cd_translations.build(title: ttl, translation: desc).save
I'm getting the error "Action Controller: incompatible character encodings: ASCII-8BIT and UTF-8".
I'm using the utf-8 encoding in my application.rb file, and I'm using the mysql2 gem.
If I remove the Registered Trade Mark character, the error goes away. But it's not really an option to strip it out of the incoming text.
I tried the solution here: https://stackoverflow.com/a/5215676/102372, but that didn't make any difference.
Stack trace:
app/controllers/users_controller.rb:28:in `block in update_cd_translations'
app/controllers/users_controller.rb:15:in `each'
app/controllers/users_controller.rb:15:in `update_cd_translations'
config/initializers/quiet_assets.rb:7:in `call_with_quiet_assets'
How can I resolve this?
It appears that ruby thinks that the encoding of the uploaded file is ascii-8bit (that is to say binary).
If you know the encoding of the file, you can use force_encoding! to change the encoding of the string (without transcoding). If you're not always going to be sure of the encoding of the file, the charguess gem can be used to guess it.
Try to add a
# -*- encoding : utf-8 -*-
at the beginnig of each file, which is included in the whole process
Related
I have a database (Mariadb) relation with a column "modified" as "bigint(10)" that represents a timestamp, I believe in unix time format. When I try to run a kafka source connector with mode "timestamp" or "timestamp+incrementing" no events are pushed into the topic. If I run only incrementing, new entries are pushed to the topic. Can someone hint to me where I configured the connector wrongly? Or does the connector not recognize timestamps in unix time format?
I tried to run a connector (retrieval based only on timestamp) with the following properties:
curl -X POST http://localhost:8083/connectors -H "Content-Type: application/json" -d '{
"name":"only_ts",
"config": {
"numeric.mapping": "best_fit",
"connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
"connection.url": "jdbc:mysql://mariadb/moodle",
"connection.user": "user",
"connection.password": "",
"topic.prefix": "only_ts_",
"mode": "timestamp",
"timestamp.column.name":"modified",
"table.whitelist":"mdl_forum_posts",
"poll.intervals.ms": 10000
}
}'
I would expect to see entries from "mdl_forum_posts" to be pushed into a kafka topic "only_ts_mdl_forum_posts" whenever I create an entry or update an entry. However, using this connector, nothing happens.
If I use only mode "incrementing" this works fine and as expected. But to get DB UPDATES too, I need to add the mode timestamp.
Output for "describe mdl_forum_posts"
+---------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-----+---------+----------------+
| id | bigint(10) | NO | PRI | NULL | auto_increment |
| discussion | bigint(10) | NO | MUL | 0 | |
| parent | bigint(10) | NO | MUL | 0 | |
| userid | bigint(10) | NO | MUL | 0 | |
| created | bigint(10) | NO | MUL | 0 | |
| modified | bigint(10) | NO | | 0 | |
| mailed | tinyint(2) | NO | MUL | 0 | |
| subject | varchar(255) | NO | | | |
| message | longtext | NO | | NULL | |
| messageformat | tinyint(2) | NO | | 0 | |
| messagetrust | tinyint(2) | NO | | 0 | |
| attachment | varchar(100) | NO | | | |
| totalscore | smallint(4) | NO | | 0 | |
| mailnow | bigint(10) | NO | | 0 | |
| deleted | tinyint(1) | NO | | 0 | |
+---------------+--------------+------+-----+---------+----------------+
And output for "show create table moodle.mdl_forum_posts;":
| mdl_forum_posts | CREATE TABLE mdl_forum_posts (
id bigint(10) NOT NULL AUTO_INCREMENT,
discussion bigint(10) NOT NULL DEFAULT '0',
parent bigint(10) NOT NULL DEFAULT '0',
userid bigint(10) NOT NULL DEFAULT '0',
created bigint(10) NOT NULL DEFAULT '0',
modified bigint(10) NOT NULL DEFAULT '0',
mailed tinyint(2) NOT NULL DEFAULT '0',
subject varchar(255) NOT NULL DEFAULT '',
message longtext NOT NULL,
messageformat tinyint(2) NOT NULL DEFAULT '0',
messagetrust tinyint(2) NOT NULL DEFAULT '0',
attachment varchar(100) NOT NULL DEFAULT '',
totalscore smallint(4) NOT NULL DEFAULT '0',
mailnow bigint(10) NOT NULL DEFAULT '0',
deleted tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (id),
KEY mdl_forupost_use_ix (userid),
KEY mdl_forupost_cre_ix (created),
KEY mdl_forupost_mai_ix (mailed),
KEY mdl_forupost_dis_ix (discussion),
KEY mdl_forupost_par_ix (parent)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8 COMMENT='All posts are stored in this table' |
An example entry in the column "modified" is:
select modified from mdl_forum_posts;
1557487199
It is a timestamp in unix time as following seems to show:
select from_unixtime(modified) from mdl_forum_posts;
2019-05-10 11:19:59
The relevant log concerning the relevant connector (only timestamp) seems to show some query ?
kafka-connect_1 | [2019-05-10 11:48:47,434] DEBUG TimestampIncrementingTableQuerier{table="moodle"."mdl_forum_posts", query='null', topicPrefix='only_ts_', incrementingColumn='', timestampColumns=[modified]} prepared SQL query: SELECT * FROM `moodle`.`mdl_forum_posts` WHERE `moodle`.`mdl_forum_posts`.`modified` > ? AND `moodle`.`mdl_forum_posts`.`modified` < ? ORDER BY `moodle`.`mdl_forum_posts`.`modified` ASC (io.confluent.connect.jdbc.source.TimestampIncrementingTableQuerier)
kafka-connect_1 | [2019-05-10 11:48:47,435] DEBUG Resetting querier TimestampIncrementingTableQuerier{table="moodle"."mdl_forum_posts", query='null', topicPrefix='only_ts_', incrementingColumn='', timestampColumns=[modified]} (io.confluent.connect.jdbc.source.JdbcSourceTask)
I had the same problem. The only workaround for me was as mentioned here: https://github.com/confluentinc/kafka-connect-jdbc/issues/566.
It means that timestamp mode for unix timestamp (bigint) column can be used together with a custom query. You only need to use your own where clause. For example in your case, it could be something like:
SELECT id
FROM mdl_forum_posts
WHERE to_timestamp(modified/1000) > ? AND to_timestamp(modified/1000) < ? ORDER BY modified ASC
--
to_timestamp is a date conversion function in your DB dialect. And please note -- that allows to comment autogenerated where clause.
If you are using newer versions, you can use the "timestamp.granularity" option: which would allow you to work with other timestamp data types.
How to save time.Now() in mysql table, column name as created_at timestamp null.
I am getting error :
Error:Error 1292: Incorrect datetime value: '2017-08-05 19:06:14.190 +0000' for column 'created_at' at row 1
More Information as asked :- ( I am using fragmenta cms, so all reference code with their line number is given below )
Table schema :-
mysql> describe users;
+----------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| created_at | timestamp | YES | | NULL | |
| updated_at | timestamp | YES | | NULL | |
| status | int(11) | YES | | NULL | |
| role | int(11) | YES | | NULL | |
| name | varchar(250) | YES | | NULL | |
| email | varchar(250) | YES | | NULL | |
| title | varchar(250) | YES | | NULL | |
| summary | text | YES | | NULL | |
| text | text | YES | | NULL | |
| image_id | int(11) | YES | | NULL | |
| password_hash | varchar(250) | YES | | NULL | |
| password_reset_token | text | YES | | NULL | |
| password_reset_at | timestamp | YES | | NULL | |
+----------------------+--------------+------+-----+---------+----------------+
Code that is running it to save :-
At line no. 62 here ( https://github.com/fragmenta/fragmenta-cms/blob/master/src/pages/actions/setup.go#L62 )
It calls code
user := users.New()
in Line no. 51 at file here ( https://github.com/fragmenta/fragmenta-cms/blob/master/src/users/query.go#L51 )
New() function is setup.
Which is like :-
func New() *User {
user := &User{}
user.CreatedAt = time.Now()
user.UpdatedAt = time.Now()
user.TableName = TableName
user.KeyName = KeyName
user.Status = status.Draft
return user
}
and their connecting / mysql opening pattern is located here ( https://github.com/fragmenta/query/blob/master/adapters/database_mysql.go#L23 ) .
There is a bug in https://github.com/fragmenta/query. The TimeString method in query/adapters/database.go is not valid for all DBMS adapters.
// TimeString - given a time, return the standard string representation
func (db *Adapter) TimeString(t time.Time) string {
return t.Format("2006-01-02 15:04:05.000 -0700")
}
It's not valid for a MySQL timestamp: MySQL 5.7 Reference Manual, 11.3.1 The DATE, DATETIME, and TIMESTAMP Types. The MySQL TimeString method in query/adapters/database_mysql.go should be:
// TimeString - given a time, return the MySQL standard string representation
func (db *MysqlAdapter) TimeString(t time.Time) string {
return t.Format("2006-01-02 15:04:05.999999")
}
You are trying to insert it using a string query.go:36:
now := query.TimeString(time.Now().UTC())
that is generated by the package that you are using database.go:59:
return t.Format("2006-01-02 15:04:05.000 -0700")
MySQL expects it to be in the pattern of yyyy-MM-dd hh:mm:ss, use the following snippet to apply the pattern to you current Time.time object:
now := time.Now().UTC().Format("2006-01-02 03:04:05")
Anyway, why not to use the SQL function NOW() when inserting the record ?
I have three tables:
- Venue
- Space (belongs to Venue)
- Included Space (belongs to Space)
I receive the id of a Venue in the route and return all the related spaces that I know have Included Spaces(a field called num_included_spaces__c on the Space record that maintains a count of its children). Now that I have all the related parent Spaces for that Venue, I need to find all of the Included Spaces for them.
An Included Space is still a Space, it just happens to have a parent that resides in the same table. I'm trying to turn this:
Venue = Rockdog
- Space = Upstairs
- Space = Media Room
- Space = Courtyard
- Space = Downstairs
- Space = Front Patio
- Space = Indoor Bar
Into this:
Venue = Rockdog
- Space = Upstairs
-- Included Space = Media Room
-- Included Space = Courtyard
- Space = Downstairs
-- Included Space = Front Patio
-- Included Space = Indoor Bar
The Included Spaces table has belongs_to__c and space__c as fields, where belongs_to__c is the id of the parent space and space__c is the id of the child. So i'm looking to find all the Included Spaces where belongs_to_c matches the id of any #spaces returned below
#sub_spaces = Space.where("venue__c = ? AND num_included_spaces__c = ?", params[:venue],0)
#spaces = Space.where("venue__c = ? AND num_included_spaces__c > ?", params[:venue],0)
How would I write this Active Record Query for #included_spaces?
my database schema.
mysql> describe venues;
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | YES | | NULL | |
| created_at | datetime | NO | | NULL | |
| updated_at | datetime | NO | | NULL | |
+------------+--------------+------+-----+---------+----------------+
4 rows in set (0,00 sec)
mysql> describe spaces;;
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | YES | | NULL | |
| venue_id | int(11) | YES | | NULL | |
| created_at | datetime | NO | | NULL | |
| updated_at | datetime | NO | | NULL | |
+------------+--------------+------+-----+---------+----------------+
5 rows in set (0,00 sec)
ERROR:
No query specified
mysql> describe included_spaces;;
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | YES | | NULL | |
| space_id | int(11) | YES | | NULL | |
| created_at | datetime | NO | | NULL | |
| updated_at | datetime | NO | | NULL | |
+------------+--------------+------+-----+---------+----------------+
5 rows in set (0,00 sec)
ERROR:
No query specified
Below function will somehow give you the result you need (in console ofcourse ) however it's not a good solution - it queries database more than needed. However it is the easy -))
def foo id
v = Venue.find(id)
puts v.name
v.spaces.each do |space|
puts space.name
space.included_spaces.each do |spis|
puts spis.name
end
end
end
You can also try a more complex query sth like,
mysql> SELECT spaces.name, included_spaces.name FROM `spaces` INNER JOIN `venues` ON `venues`.`id` = `spaces`.`venue_id` INNER JOIN `included_spaces` ON `included_spaces`.`space_id` = `spaces`.`id` WHERE `spaces`.`venue_id` = 1
-> ;
+------------+-----------+
| name | name |
+------------+-----------+
| Upstairs, | Front |
| Upstairs, | Patio, |
| Upstairs, | Indoor |
| Upstairs, | Bar |
| Downstairs | Media |
| Downstairs | Room, |
| Downstairs | Courtyard |
+------------+-----------+
7 rows in set (0,00 sec)
which should be translated to active record as
Space.joins(:venue)
.joins(:included_spaces)
.where(venue_id: 1)
.select('spaces.name, included_spaces.name')
I have a code to cek if the user is exist. I return some data to display to user. This is my code
Model
public function cek_exist_master($no_eir){
$this->db->select('REPAIR_ESTIMATE_ID, EIR_REF');
$this->db->where('EIR_REF', $no_eir);
$query = $this->db->get('tb_master_repair_estimate');
return $query;
}
CONTROLLER
$cek_master =$this->m_surveyor->cek_exist_master($this->input->post('EIR_REF'));
echo $cek_master->row()->REPAIR_ESTIMATE_ID;
And this is the database looked like :
mysql> desc tb_master_repair_estimate;
+--------------------+-----------------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+-----------------------+------+-----+-------------------+-----------------------------+
| REPAIR_ESTIMATE_ID | int(11) | NO | PRI | NULL | auto_increment |
| EIR_REF | varchar(30) | YES | UNI | NULL | |
| NO_TANK | int(11) | YES | MUL | NULL | |
| COSTUMER_ID | int(11) | YES | MUL | NULL | |
| TANK_ID | int(11) | YES | MUL | NULL | |
+--------------------+-----------------------+------+-----+-------------------+-----------------------------+
24 rows in set (0.01 sec)
CI gives me an error like this,
A PHP Error was encountered
Severity: Notice
Message: Trying to get property of non-object
Filename: surveyor/c_surveyor.php
Any help it so appreciate
Update,
Sorry, have bad copy paste on stackoverflow. I still get those error untill Now
You forgot the $ sign in your controller. It should be:
$cek_master = $this->m_surveyor->cek_exist_master($this->input->post('EIR_REF'));
Maybe a typo?
$cek_master = this->m_surveyor->...
Should be
$cek_master = $this->m_surveyor->...
First check that you load your model "m_surveyor" into controller.
CONTROLLER.. Change in controller code
$cek_master =this->m_surveyor->cek_exist_master($this->input->post('EIR_REF'));
echo is_object($cek_master->row()) ? $cek_master->row()->REPAIR_ESTIMATE_ID : '';
I have these DB tables:
taggings
id | integer | not null default nextval('taggings_id_seq'::regclass) | plain |
tag_id | integer | | plain |
taggable_id | integer | | plain |
taggable_type | character varying(255) | | extended |
tagger_id | integer | | plain |
tagger_type | character varying(255) | | extended |
context | character varying(128) | | extended |
created_at | timestamp without time zone | | plain |
tags
id | integer | not null default nextval('tags_id_seq'::regclass) | plain |
name | character varying(255) | | extended |
These tables are created by the acts-as-taggable-on gem.
I am trying to get the count of most used tags that are stored in the table taggings, but I have no idea how to get them...
Could anyone give me some tip or link, how to get that?