Fix RVM and Openssl Issue

Tags

mac, osx, personal, rails, ruby, security, software-development, Technology

Today while running rails console, I got a weird error.

/Users/chandra/.rvm/gems/ruby-2.3.1/gems/activesupport-5.0.0.1/lib/active_support/dependencies.rb:293:in `require': dlopen(/Users/chandra/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/x86_64-darwin14/readline.bundle, 9): Library not loaded: /usr/local/opt/readline/lib/libreadline.6.dylib (LoadError)
Referenced from: /Users/chandra/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/x86_64-darwin14/readline.bundle
Reason: image not found - /Users/chandra/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/x86_64-darwin14/readline.bundle

While debugging and performing a google search, I came through a solution to reinstall the ruby version. I followed the same.

Oops, I have to face another glitch

Error running '__rvm_make -j 1',
showing last 15 lines of /Users/chandra/.rvm/log/1472071369_ruby-2.3.1/make.log
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ossl_ssl.c:18:35: note: expanded from macro 'numberof'
#define numberof(ary) (int)(sizeof(ary)/sizeof((ary)[0]))
^~~~~
ossl_ssl.c:2267:21: error: invalid application of 'sizeof' to an incomplete type 'const struct (anonymous struct at ossl_ssl.c:85:14) []'
for (i = 0; i < numberof(ossl_ssl_method_tab); i++) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ossl_ssl.c:18:35: note: expanded from macro 'numberof'
#define numberof(ary) (int)(sizeof(ary)/sizeof((ary)[0]))
^~~~~
6 warnings and 9 errors generated.
make[2]: *** [ossl_ssl.o] Error 1
make[1]: *** [ext/openssl/all] Error 2
make: *** [build-ext] Error 2
++ return 2
There has been an error while running make. Halting the installation.

I got freaked up(what’s happening?). What changes I have made recently to the system? I try to remember hard but didn’t remember anything except fixing some SSL CERT issue.

Again I research on ossl_ssl and found that it’s OpenSSL issue. And now I start debugging the system and found that it’s OpenSSL symlink issue. I stopped here and to get proper remedy I follow these steps

Brew update

cd $(brew --repo); git fetch; git reset --hard origin/master; brew update

Uninstall OpenSSL(ignoring dependencies)

brew uninstall --ignore-dependencies openssl
brew install openssl
brew link --overwrite openssl —force

Export OpenSSL provided flags (LDFLAGS and CPPFLAGS)

export LDFLAGS=-L/usr/local/opt/openssl/lib
export CPPFLAGS=-I/usr/local/opt/openssl/include

Cleanup and get RVM master
There was an issue with RVM https://github.com/rvm/rvm/issues/3748

brew install ruby-build
rvm cleanup all
rvm get head

ReInstall Ruby and bundler

rvm install 2.3.1
gem install bundler

~/c/t/issue git⭠master ❯❯❯ rails c ✭
Running via Spring preloader in process 26404
Loading development environment (Rails 5.0.0.1)
gem install awesome_print #

Now, everything is working as expected, it’s time for coffee.

Replication(Master/Slave) of PostgreSQL database

Tags

database, devops, postgres, postgresql, server, ubuntu

Before you proceed this handy procedure, be sure you take backup of the databases.

 pg_dumpall > dump.sql

Prerequisite:

1. Two Ubuntu(14.04) instances:
   Master(187.12.0.1)
   Slave(187.12.0.2)
2. PostgreSQL 9.4.x

Steps to be followed on MASTER:

Step 1: Create a replication user on the master with REPLICATION permissions:

sudo -u postgres createuser -U postgres replicator -P -c 5 --replication

-P prompts you for the new user’s password.
-c sets a limit for the number of connections for the new user. The value 5 is sufficient for replication purposes.
–replication grants the REPLICATION privilege to the user named replication.

Step 2: Configure master for streaming replication

vi /etc/postgresql/9.4/main/postgresql.conf

listen_address = ''
wal_level = hot_standby
max_wal_senders = 3
checkpoint_segments = 8
wal_keep_segments = 8

Here each wal_keep_segments is of 16MB. If you expect your database to have more than 128MB of changes in the time it will take to make a copy of it across the network to your slave, or in the time you expect your slave to be down for maintenance or something, then consider increasing those values.
Step 3: Configure master to allow the connection from the slave

vi /etc/postgresql/9.4/main/pg_hba.conf

#if your server is not ssl ready, use hostnossl

hostssl replication replicator 187.12.0.2 md5

Or
hostnossl replication replicator 187.12.0.2 md5

here hostssl means this host can only connect via SSL.

Restart the server now.

Steps to be followed on SLAVE:

Step 1: Configure master for streaming replication

vi /etc/postgresql/9.4/main/postgresql.conf

wal_level = hot_standby
max_wal_senders = 3
checkpoint_segments = 8
wal_keep_segments = 8
hot_standby = on

No change required for pg_hba.conf

Step 2: Copy the database from master and begin replication

#Stop server

sudo service postgresql stop

#Clean up old cluster directory
ATTENTION: script is going to delete the old database cluster on your slave.

sudo -u postgres rm -rf /var/lib/postgresql/9.4/main

#Start base backup as replicator with pg_basebackup command

sudo -u postgres pg_basebackup -h 187.12.0.1 -D /var/lib/postgresql/9.4/main -U replicator -v -P

Create recovery.conf file and update the content

sudo -u postgres bash -c "cat > /var/lib/postgresql/9.4/main/recovery.conf <<- _EOF1_
standby_mode = 'on'
primary_conninfo = 'host=187.12.0.1 port=5432 user=replicator password=PASSWORD sslmode=require'
trigger_file = '/tmp/postgresql.trigger'
_EOF1_
"

NOTE: remove sslmode=require is ssl is not configured for the server.

#Start the PostgreSQL server

sudo service postgresql start

 

Great, we have completed the replication of PostgreSQL database. To make sure everything working as expected execute and watch:

sudo -u postgres psql -x -c "select * from pg_stat_replication;"

Happy learning.

Fix MySql group_by after upgrading mysql

Tags

active-record, database, devops, ei-capitan, mac, mysql, osx, rails, ruby, server, software-development

MySql version 5 contains breaking changes among which default SQL mode is one of them.

The new default SQL mode includes these modes:

• ONLY_FULL_GROUP_BY,
• STRICT_TRANS_TABLES
• NO_ZERO_IN_DATE
• NO_ZERO_DATE
• ERROR_FOR_DIVISION_BY_ZERO
• NO_AUTO_CREATE_USER
• NO_ENGINE_SUBSTITUTION.

Brief on new changed modes

1. ONLY_FULL_GROUP_BY and STRICT_TRANS_TABLES modes were added in MySQL 5.7.5
2. NO_AUTO_CREATE_USER mode was added in MySQL 5.7.7
3. ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, and NO_ZERO_IN_DATE modes were added in MySQL 5.7.8

You can find different SQl modes here

 
The ONLY_FULL_GROUP_BY mode means :-

Reject queries for which the select list, HAVING condition, or ORDER BY list refer to nonaggregated columns that are neither named in the GROUP BY clause nor are functionally dependent on (uniquely determined by) GROUP BY columns.

Example
To give you an example of what that means here is a simple SQL query that will fail with the above SQL mode.

 SELECT first_name, IF(last_name IS NULL, 'DUMMY', last_name) as last_name from users GROUP BY first_name

SELECT first_name, IF(last_name IS NULL, ‘DUMMY’, last_name) as last_name from users GROUP BY first_name

This query will give you the following error: –

Mysql2::Error: Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column ‘tmp_tbl.value’ which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by: SELECT value, table_id, count(*) as tables_count
FROM (
SELECT value, table_id
FROM noker_tables
WHERE noker_id = 8
ORDER BY id DESC
) as tmp_tbl
GROUP BY table_id
Completed 500 Internal Server Error in 11.5ms

 

Solution
If you are like me and have used Homebrew to install/upgrade Mysql, then you can do the following steps to fix the issue.

Copy the default my-default.cnf to /etc/my.cnf

sudo cp $(brew --prefix mysql)/support-files/my-default.cnf /etc/my.cnf

sudo vi /etc/my.cnf +31

Now set sql_mode it to this(you can find this near line 31) :-

 sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

After writing my.cnf, restart your MySQL server.

**But if you are trying something in MySQL console use this commands

set session sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
set global sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';

 

Now your queries will start working again. Happy debugging

Postmortem of “400 Bad Request – request header or cookie too large”

Tags

devops, nginx, ruby, server, software-development

 

Issue:

400 Bad Request – request header or cookie too large

 

---

Solution:

Increase the size of large_client_header_buffers of server/http block in Nginx

 server {
   # ...
   large_client_header_buffers 4 32k; # or more according to the need
   # ...
  }

Synopsis :

Sets the maximum number and size of buffers used for reading large client request header. A request line cannot exceed the size of one buffer, or the 414 (Request-URI Too Large) error is returned to the client.
A request header field cannot exceed the size of one buffer as well, or the 400 (Bad Request) error is returned to the client. Buffers are allocated only on demand. By default, the buffer size is equal to 8K bytes. If after the end of request processing a connection is transitioned into the keep-alive state, these buffers are released.

By the way, the default buffer number and size is 4 and 8k, so the header must be the one that’s over 8192 bytes.
In this case, all those cookies (which combine to one header) are well over the limit.

 

For more details please refer Nginx Site