Warning: Undefined variable $currentPage in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 209

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "tags" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 313

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "topper" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 317

Warning: Undefined array key "topperalt" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 318

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Warning: Undefined array key "small-label" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 314

Warning: Undefined array key "summary" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 319

Blog Posts


Warning: Undefined variable $folderMissingAlert in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 836

Warning: Undefined variable $enablePrettyURLS in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 912

Warning: Undefined variable $urlSearch in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1318

Warning: Undefined variable $foundPost in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1330

Warning: Undefined variable $foundPost in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1337

Warning: Undefined variable $foundPost in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1330

Warning: Undefined variable $foundPost in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1337

Warning: Undefined variable $foundPost in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1330

Warning: Undefined variable $foundPost in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1337

Warning: Undefined variable $foundPost in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1330

Warning: Undefined variable $foundPost in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1337

Warning: Undefined array key "extension" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 110

Warning: Undefined array key "extension" in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 110

Warning: Undefined variable $isTheFeaturedTitleInUse in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1400

Warning: Undefined variable $isTheFeaturedTitleInUse in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1416

The new server is even quicker than I thought

Written by: | Posted on: | Category:

This is just a quick look at a process we run for checking on Northern Lights activity, The gist of it is we download a file from a NOAA site, convert it from JSON to CSV, do a quick check to make sure the data is new (dtCreated and dtValid), then load it into the database. The NOAA file has 65,160 records with 5 fields per record, so it's a decent chunk of data.

Here's a log file from the old server:

2021/06/01 00:20:01 INFO> CPU.pm:194 - CPU load: 1.58 <= 8.00
2021/06/01 00:20:01 INFO> AuroraPercentDB_Load.pl:134 - Getting https://services.swpc.noaa.gov/json/ovation_aurora_latest.json
2021/06/01 00:20:06 INFO> AuroraPercentDB_Load.pl:149 - dt Created: 2021-06-01 05:11:00
2021/06/01 00:20:06 INFO> AuroraPercentDB_Load.pl:150 - dt Valid:   2021-06-01 06:34:00

And here's a log file from the new server:

2021/06/28 22:35:23 INFO> CPU.pm:194 - CPU load: 1.50 <= 8.00
2021/06/28 22:35:23 INFO> AuroraPercentDB_Load.pl:134 - Getting https://services.swpc.noaa.gov/json/ovation_aurora_latest.json
2021/06/28 22:35:23 INFO> AuroraPercentDB_Load.pl:149 - dt Created: 2021-06-29 03:26:00
2021/06/28 22:35:23 INFO> AuroraPercentDB_Load.pl:150 - dt Valid:   2021-06-29 04:50:00

Notice the gap in time between the 2nd and 3rd line of the first run (00:20:01 to 00:20:06) is five seconds and the gap for the second run (22:35:23 to 22:35:23) is zero seconds. There are really only two things happening in that time, downloading the file and parsing the JSON format into a structure the program can more easily understand. The time required to download the file shouldn't change from old to new server since they are both on the same network. So, the new server is parsing the JSON format much quicker for some reason.

I expected things to be faster, but that's much better than I had hoped.


Warning: Undefined variable $foundNextPost in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1337

Warning: Undefined variable $urlCategory in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1465

Warning: Undefined variable $urlCategory in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1579

Warning: Undefined variable $urlSearch in /opt/homebrew/var/www/customvisuals.com/blog/index.php on line 1697

© 2003-2021      Custom Visuals, LLC      Privacy Policy      Sitemap