chronyc

Support forum for the ViciBox ISO Server Install and ISO LiveCD Demo

Moderators: enjay, williamconley, Staydog, mflorell, MJCoate, mcargile, Kumba

chronyc

Postby kashinc » Tue May 26, 2020 11:53 am

I have been running the new vicibox 9.0.2 and have seen high system usage on a few machines, the one machine that is bothering me most is my master database server. Chronyc is using 100% cpu on this server... When I was running 9.0.0 I always had low load averages, has anyone else had any similar issues or know how to fix this?
kashinc
 
Posts: 57
Joined: Thu Apr 23, 2015 12:04 pm

Re: chronyc

Postby carpenox » Tue May 26, 2020 7:13 pm

u sure its chrony? mysqld is pretty high load using a lot of Mem%, but not over 50% ever, but i run a little diff, im on asterisk 13.32.0 instead of 29.
2 x Intel Xeon X3450 at 2.66GHz | 16GB DDR4
ViciBox v9.0.3 | Version: 2.14-772a | BUILD: 201004-1045 | SVN Version: 3304 | DB Schema Version: 1608 | Asterisk 13.34.0
http://www.CyburityLLC.com -: 844-PC-SATA-2 - :- www.contactcentersRus.com
carpenox
 
Posts: 698
Joined: Wed Apr 08, 2020 2:02 am
Location: Coral Springs, FL

Re: chronyc

Postby Kumba » Mon Jun 01, 2020 4:33 pm

Chronyc is it's own monolithic piece of software. In some random searches google mentions things like network changes (DHCP) as causing issues with Chrony. If you haven't already, you can try running package updates to see if it pulls anything in.

Other then that, I really don't have a clue. I'd have to be able to duplicate the issue and I have never ran across chrony trying to peg a CPU.
Kumba
 
Posts: 871
Joined: Tue Oct 16, 2007 11:44 pm
Location: Florida

Re: chronyc

Postby kashinc » Thu Jun 04, 2020 9:13 pm

This problem seems to be related to a AMD chips... Both webserver and database server were both maxed out at 100% cpu usage.

I installed the latest Kernel 5.6 and the issue is now gone, hope this helps for anyone else running into this problem.
kashinc
 
Posts: 57
Joined: Thu Apr 23, 2015 12:04 pm

Re: chronyc

Postby Kumba » Fri Jun 05, 2020 4:44 pm

Kernel 5.6 isn't a default option on OpenSuSE v.15.1. Were you already running current updates on ViciBox v.9 before doing that update?

Also, what AMD CPUs were they?
Kumba
 
Posts: 871
Joined: Tue Oct 16, 2007 11:44 pm
Location: Florida

Re: chronyc

Postby kashinc » Sun Jun 07, 2020 3:21 pm

Kumba wrote:Kernel 5.6 isn't a default option on OpenSuSE v.15.1. Were you already running current updates on ViciBox v.9 before doing that update?

Also, what AMD CPUs were they?


Yes, I was running current updates... it happens on some of the servers its very strange...

it shows as /usr/bin/chronyc -a -n -m tracking

it hogs 100% of system usage... only happens on some installs and is not only AMD specific...

upgrading the kernel to this version 5.7.0-3.gad96a07-default

will fix the chronyc issue, however this causes issues with dahdi no longer working and you will have to manually compile it for it to work properly... not sure if this is the best route... just trying to give you some feedback Kumba so we can figure it out and make progress, I enjoy giving back to the community whenever possible.
kashinc
 
Posts: 57
Joined: Thu Apr 23, 2015 12:04 pm

Re: chronyc

Postby kashinc » Sun Jun 07, 2020 9:36 pm

kashinc wrote:
Kumba wrote:Kernel 5.6 isn't a default option on OpenSuSE v.15.1. Were you already running current updates on ViciBox v.9 before doing that update?

Also, what AMD CPUs were they?


Yes, I was running current updates... it happens on some of the servers its very strange...

it shows as /usr/bin/chronyc -a -n -m tracking

it hogs 100% of system usage... only happens on some installs and is not only AMD specific...

upgrading the kernel to this version 5.7.0-3.gad96a07-default

will fix the chronyc issue, however this causes issues with dahdi no longer working and you will have to manually compile it for it to work properly... not sure if this is the best route... just trying to give you some feedback Kumba so we can figure it out and make progress, I enjoy giving back to the community whenever possible.


also I have noticed that this doesn't happen on install, or even after zypper ref and zypper up reboot. However after i do vicibox-install this starts happening.
kashinc
 
Posts: 57
Joined: Thu Apr 23, 2015 12:04 pm

Re: chronyc

Postby SPAMSAM » Sat Aug 08, 2020 4:28 pm

This issue happens to me on install prior to vicibox-install on 9.0.3

The issue seems to have two components, 1.) chronyd crashes on startup with faiure to lock /var/run/chrony-helper and 2.) 100% CPU usage on one core by /usr/bin/chronyc -a -n -m tracking.

Issue 1 can be resolved by running the following:
Code: Select all
rm -f /var/run/chrony-helper/lock
service chronyd restart


Note that if the system reboots the error will reoccur. Also once chronyd is up and running again it keeps using 100% CPU. So not a fix.

Any suggestions are welcome as the issue seems to be hardware-agnostic.
SPAMSAM
 
Posts: 42
Joined: Tue Jan 17, 2017 4:00 am

Re: chronyc

Postby Kumba » Fri Sep 11, 2020 2:21 am

Whatever is broken with chrony is beyond me. I have found nothing to explain it's behavior. Some servers it works like a champ, some it dies. I think it might be related to network connectivity on start-up but I haven't been able to definitively test that.

My solution is to burn it with fire and install ntp. Here's how you do that:

Code: Select all
service chronyd stop
killall chronyc
zypper rm chrony  (enter y when prompted)
zypper in ntp
systemctl enable ntpd.service
echo "restrict -4 default kod notrap nomodify nopeer noquery" >> /etc/ntp.conf
echo "restrict -6 default kod notrap nomodify nopeer noquery" >> /etc/ntp.conf
echo "server 127.127.1.0" >> /etc/ntp.conf
echo "fudge 127.127.1.0  stratum 10" >> /etc/ntp.conf
echo "server 0.opensuse.pool.ntp.org  iburst" >> /etc/ntp.conf
echo "server 1.opensuse.pool.ntp.org  iburst" >> /etc/ntp.conf
echo "server 2.opensuse.pool.ntp.org  iburst" >> /etc/ntp.conf
echo "server 3.opensuse.pool.ntp.org  iburst" >> /etc/ntp.conf
service ntpd start


That should fix chrony permanently. The down side is the default ntp configuration doesn't allow it to function as a time source, but that's probably not a big issue unless you have large large clusters.
Kumba
 
Posts: 871
Joined: Tue Oct 16, 2007 11:44 pm
Location: Florida

Re: chronyc

Postby carpenox » Fri Sep 11, 2020 10:44 am

good information to have, seems to be a problem with a lot of new installs. i havent had this problem on my r240's but i know a lot of people have
2 x Intel Xeon X3450 at 2.66GHz | 16GB DDR4
ViciBox v9.0.3 | Version: 2.14-772a | BUILD: 201004-1045 | SVN Version: 3304 | DB Schema Version: 1608 | Asterisk 13.34.0
http://www.CyburityLLC.com -: 844-PC-SATA-2 - :- www.contactcentersRus.com
carpenox
 
Posts: 698
Joined: Wed Apr 08, 2020 2:02 am
Location: Coral Springs, FL

Re: chronyc

Postby Fares MEHENI » Mon Oct 12, 2020 4:10 am

Kumba wrote:Whatever is broken with chrony is beyond me. I have found nothing to explain it's behavior. Some servers it works like a champ, some it dies. I think it might be related to network connectivity on start-up but I haven't been able to definitively test that.

My solution is to burn it with fire and install ntp. Here's how you do that:

Code: Select all
service chronyd stop
killall chronyc
zypper rm chrony  (enter y when prompted)
zypper in ntp
systemctl enable ntpd.service
echo "restrict -4 default kod notrap nomodify nopeer noquery" >> /etc/ntp.conf
echo "restrict -6 default kod notrap nomodify nopeer noquery" >> /etc/ntp.conf
echo "server 127.127.1.0" >> /etc/ntp.conf
echo "fudge 127.127.1.0  stratum 10" >> /etc/ntp.conf
echo "server 0.opensuse.pool.ntp.org  iburst" >> /etc/ntp.conf
echo "server 1.opensuse.pool.ntp.org  iburst" >> /etc/ntp.conf
echo "server 2.opensuse.pool.ntp.org  iburst" >> /etc/ntp.conf
echo "server 3.opensuse.pool.ntp.org  iburst" >> /etc/ntp.conf
service ntpd start


That should fix chrony permanently. The down side is the default ntp configuration doesn't allow it to function as a time source, but that's probably not a big issue unless you have large large clusters.



It's working now, thanks a lot Kumba .
Fares MEHENI
 
Posts: 41
Joined: Thu Oct 17, 2019 2:43 am


Return to ViciBox Server Install and Demo

Who is online

Users browsing this forum: No registered users and 13 guests