{"id":673,"date":"2020-04-11T14:33:28","date_gmt":"2020-04-11T18:33:28","guid":{"rendered":"http:\/\/www.aibistin.com\/?p=673"},"modified":"2023-03-12T16:09:22","modified_gmt":"2023-03-12T20:09:22","slug":"creating-a-simple-json-nyc-zip-code-detail-config-file-with-perl","status":"publish","type":"post","link":"https:\/\/www.aibistin.com\/?p=673","title":{"rendered":"Creating A Simple JSON NYC Zip Code Database File With Perl and MooX::Options"},"content":{"rendered":"\n\n\n\n\n<p>I found myself needing some New York City detailed Zip Code information for another script I was creating. The zip codes themselves are easy enough to find online. I needed to include more details about each zip code location.&nbsp; I created a Perl script to merge two hard coded Perl data structures, which are printed out as a very basic JSON database file.<\/p>\n<p>When creating Perl scripts with command line options, my go-to CPAN module is <a href=\"https:\/\/metacpan.org\/pod\/Getopt::Long\" target=\"_blank\" rel=\"noopener noreferrer\">Getopt::Long<\/a>. However for this script I will use <a href=\"https:\/\/metacpan.org\/pod\/MooX::Options\" target=\"_blank\" rel=\"noopener noreferrer\">MooX::Options<\/a>, as I may extract some of the methods to be used in a future Moo module.<\/p>\n<p>This will have three options, &#8216;create_zip_db&#8217;, &#8216;read_zip_db&#8217;&nbsp; and &#8216;verbose&#8217;. The &#8216;doc&#8217; attribute gives a brief description of each option. The &#8216;short&#8217; attribute specifies any aliases that can be used for each option. The is &#8216;ro&#8217; , means that the option value is immutable.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: perl; gutter: false; title: ; notranslate\" title=\"\">\noption create_zip_db =&gt; (\n    is    =&gt; 'ro',\n    short =&gt; 'new_zipdb|new_zip',\n    doc   =&gt; q\/Create a new NYC Zip, Borough, District, Town JSON file.\/,\n);\n\noption read_zip_db =&gt; (\n    is    =&gt; 'ro',\n    short =&gt; 'read_db',\n    doc   =&gt; q\/Read the NYC Zip file database.\/,\n);\n\noption verbose =&gt; ( is =&gt; 'ro', doc =&gt; 'Print details' );\n<\/pre><\/div>\n\n\n<p>There are three Moo attributes.&nbsp; Some time in the future I can put these into a separate <a href=\"https:\/\/metacpan.org\/pod\/Moo\" target=\"_blank\" rel=\"noopener noreferrer\">Moo <\/a>module.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: perl; gutter: false; title: ; notranslate\" title=\"\">\nhas db_dir =&gt; (\n    is      =&gt; 'rw',\n    isa     =&gt; Path,\n    coerce  =&gt; 1,\n    default =&gt; sub { &quot;$Bin\/..\/db&quot; }\n);\n\nhas zip_db_json_file =&gt; (\n    is      =&gt; 'lazy',\n    isa     =&gt; Path,\n    builder =&gt; sub {\n        $_&#x5B;0]-&gt;db_dir-&gt;child(&quot;zip_db.json&quot;);\n    }\n);\n\nhas zip_hash =&gt; (\n    is =&gt; 'lazy',\n    isa =&gt;\n      sub { die &quot;'zips_hash' must be a HASH&quot; unless ( ref( $_&#x5B;0] ) eq 'HASH' ) }\n    ,\n    builder =&gt; sub {\n        deserialize_file $_&#x5B;0]-&gt;zip_db_json_file;\n    }\n);\n<\/pre><\/div>\n\n\n<p>The first attribute &#8216;db_dir&#8217; specifies the future location of the JSON file. It uses <a href=\"https:\/\/metacpan.org\/pod\/Types::Path::Tiny\" target=\"_blank\" rel=\"noopener noreferrer\">Types::Path Tiny <\/a>&nbsp; to enforce this directory path as a <a href=\"https:\/\/metacpan.org\/pod\/Path::Tiny\">Path::Tiny&nbsp; <\/a>object. The &#8216;zip_db_json_file&#8217; is also a Types::Path::Tiny Path.<\/p>\n<p>The &#8216;zip_hash&#8217; is the data structure what will store the NYC Zip code, borough, district, town information. The &#8216;isa&#8217; for this attribute will ensure that it is a Perl hash.&nbsp; The &#8216;deserialize_file&#8217;&nbsp; function comes from the CPAN module, <a href=\"https:\/\/metacpan.org\/pod\/File::Serialize\" target=\"_blank\" rel=\"noopener noreferrer\">File::Serialize<\/a> , which is very useful for dumping out Perl data structures to a JSON file, or in this case slurping in a JSON file to a Perl data structure. It also handles formats other than JSON.<\/p>\n<p>Note that the &#8216;zip_hash&#8217; attribute is &#8216;lazy&#8217;.&nbsp; I&#8217;m not saying that zip codes are particularly adverse to work. This is just Moo&#8217;s way of saying, &#8220;please don&#8217;t make me do anything until I really have to&#8221;.&nbsp; That way, resources are not nu-necessarily used creating a structure that isn&#8217;t being called for.&nbsp;<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: perl; gutter: false; title: ; notranslate\" title=\"\">\n# Main\nsub run {\n    my ($self) = @_;\n    $self-&gt;create_new_zipdb_file if $self-&gt;create_zip_db;\n    $self-&gt;read_and_dump_the_db  if $self-&gt;read_zip_db;\n    say &quot;All Done!&quot;              if $self-&gt;verbose;\n}\nmain-&gt;new_with_options()-&gt;run;\n<\/pre><\/div>\n\n\n<p><a rel=\"noreferrer noopener\" href=\"Options\" target=\"_blank\">MooX::Options<\/a> has it&#8217;s own particular style for creating a &#8220;Main&#8221; function that you won&#8217;t usually see in standard Perl scripts. It may be borrowed from brian d foy&#8217;s <a rel=\"noreferrer noopener\" href=\"https:\/\/www.masteringperl.org\/2013\/08\/new-to-modules-as-programs\/\" target=\"_blank\">&#8220;Modulino&#8221;<\/a> concept. Anyway, the script is invoked by: <\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">main-&gt;new_with_options()-&gt;run;<\/pre>\n\n\n\n<p>The main &#8216;run&#8217; function will call the methods as specified by the command line options.<\/p>\n<p>To run this script from the command line.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; gutter: false; title: ; notranslate\" title=\"\">\n# To get help\n\u03bb perl bin\\create_zipdb.pl -h\nUSAGE: create_zipdb.pl &#x5B;-h] &#x5B;long options ...]\n\n    --create_zip_db  Create a new NYC Zip, Borough, District, Town JSON\n                     file.\n    --read_zip_db    Read the NYC Zip file database.\n    --verbose        Print details\n\n    --usage          show a short help message\n    -h               show a compact help message\n    --help           show a long help message\n    --man            show the manual\n\n# Create a JSON file database\n\u03bb perl bin\\create_zipdb.pl --create_zip_db --v\n\n# Read the database and dump to the terminal\n\u03bb perl bin\\create_zipdb.pl --read_zip_db\n<\/pre><\/div>\n\n\n<p>Most of the actual work of reading in the hard coded data structure and creating\/reading the JSON database file is done here:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: perl; auto-links: true; gutter: false; title: ; notranslate\" title=\"\">\nsub create_new_zipdb_file {\n    my $self          = shift;\n    my $zip_boro_dist = $self-&gt;get_raw_zip_data();\n    serialize_file $self-&gt;zip_db_json_file =&gt; $zip_boro_dist;\n    say &quot;Created a new &quot; . $self-&gt;zip_db_json_file if $self-&gt;verbose;\n}\n\nsub get_raw_zip_data {\n    my $self         = shift;\n    my %zips_to_city = %{ _get_zips_to_city() };\n    my %bdz          = %{ _get_borough_district_zips() };\n    my %zip_boro_dist;\n    for my $borough ( sort keys %bdz ) {\n        my %district = %{ $bdz{$borough} };\n        for my $district_name ( sort keys %district ) {\n            my @district_zips = @{ $district{$district_name} };\n            for my $zip ( sort @district_zips ) {\n                my ( $city, $county ) = split \/,\/, $zips_to_city{$zip};\n                $county =\n                    $borough eq 'Brooklyn' ? 'Kings'\n                  : $borough eq 'Bronx'    ? 'Bronx'\n                  : 'New York'\n                  unless $county;\n\n                $zip_boro_dist{$zip} = {\n                    borough  =&gt; $borough,\n                    district =&gt; $district_name,\n                    city     =&gt; $city,\n                    county   =&gt; $county,\n                };\n            }\n        }\n    }\n    return \\%zip_boro_dist;\n}\n\nsub read_and_dump_the_db {\n    my $self         = shift;\n    my $location_rec = $self-&gt;zip_hash;\n    dump $location_rec;\n}\n<\/pre><\/div>\n\n\n<p>Method &#8216;get_raw_zip_data&#8217; grabs the two hard coded data structures and merges them. It makes a few little adjustments.&nbsp; It is called by &#8216;create_new_zipdb_file which uses the &#8216;serialize_file&#8217; function from&nbsp; <a href=\"https:\/\/metacpan.org\/pod\/File::Serialize\" target=\"_blank\" rel=\"noopener noreferrer\">File::Serialize<\/a> to dump the the Perl data structure in JSON format to the output JSON file.<\/p>\n<p>Method &#8216;read_and_dump_the_db&#8217; just reads this JSON file into the &#8216;zip_hash&#8217; and dumps the contents to the console.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: jscript; gutter: false; title: ; notranslate\" title=\"\">\n   &quot;10022&quot; : {\n      &quot;borough&quot; : &quot;Manhattan&quot;,\n      &quot;city&quot; : &quot;New York&quot;,\n      &quot;county&quot; : &quot;New York&quot;,\n      &quot;district&quot; : &quot;Gramercy Park and Murray Hill&quot;\n   },\n   &quot;10023&quot; : {\n      &quot;borough&quot; : &quot;Manhattan&quot;,\n      &quot;city&quot; : &quot;New York&quot;,\n      &quot;county&quot; : &quot;New York&quot;,\n      &quot;district&quot; : &quot;Upper West Side&quot;\n   },\n   ...\n     &quot;10314&quot; : {\n      &quot;borough&quot; : &quot;Staten Island&quot;,\n      &quot;city&quot; : &quot;Staten Island&quot;,\n      &quot;county&quot; : &quot;Richmond&quot;,\n      &quot;district&quot; : &quot;Mid-Island&quot;\n   },\n   &quot;10451&quot; : {\n      &quot;borough&quot; : &quot;Bronx&quot;,\n      &quot;city&quot; : &quot;Bronx&quot;,\n      &quot;county&quot; : &quot;Bronx&quot;,\n      &quot;district&quot; : &quot;High Bridge and Morrisania&quot;\n   },\n   ...\n  &quot;11426&quot; : {\n      &quot;borough&quot; : &quot;Queens&quot;,\n      &quot;city&quot; : &quot;Bellerose&quot;,\n      &quot;county&quot; : &quot;Queens&quot;,\n      &quot;district&quot; : &quot;Southeast Queens&quot;\n   },\n   &quot;11427&quot; : {\n      &quot;borough&quot; : &quot;Queens&quot;,\n      &quot;city&quot; : &quot;Queens Village&quot;,\n      &quot;county&quot; : &quot;Queens&quot;,\n      &quot;district&quot; : &quot;Southeast Queens&quot;\n   },\n   &quot;11428&quot; : {\n      &quot;borough&quot; : &quot;Queens&quot;,\n      &quot;city&quot; : &quot;Queens Village&quot;,\n      &quot;county&quot; : &quot;Queens&quot;,\n      &quot;district&quot; : &quot;Southeast Queens&quot;\n   },\n<\/pre><\/div>\n\n\n<p>The complete script can be found here <a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/aibistin\/covid\/blob\/master\/bin\/create_zipdb.pl\" target=\"_blank\"><strong>create_zipdb.pl<\/strong><\/a><\/p>\n\n\n\n<!--nextpage-->\n\n\n\n\n<ol class=\"has-avatars has-dates has-excerpts wp-block-latest-comments\"><li class=\"wp-block-latest-comments__comment\"><img alt='' src='https:\/\/secure.gravatar.com\/avatar\/4dd11a0239bf3aadc7467a398f6a2843300346c974826db561b1dbdf874126d4?s=48&#038;d=mm&#038;r=g' srcset='https:\/\/secure.gravatar.com\/avatar\/4dd11a0239bf3aadc7467a398f6a2843300346c974826db561b1dbdf874126d4?s=96&#038;d=mm&#038;r=g 2x' class='avatar avatar-48 photo wp-block-latest-comments__comment-avatar' height='48' width='48' \/><article><footer class=\"wp-block-latest-comments__comment-meta\"><a class=\"wp-block-latest-comments__comment-author\" href=\"http:\/\/www.carryonmoving.com\">Austin<\/a> on <a class=\"wp-block-latest-comments__comment-link\" href=\"https:\/\/www.aibistin.com\/?p=23#comment-4424\">Upload and Email a photo using Dancer2 with Twitter Bootstrap<\/a><time datetime=\"2017-03-12T15:34:19-04:00\" class=\"wp-block-latest-comments__comment-date\">March 12, 2017<\/time><\/footer><div class=\"wp-block-latest-comments__comment-excerpt\"><p>Thanks for pointing this out Dave. I just made that alteration. Oh, and congratulations on the dubious honour of posting&hellip;<\/p>\n<\/div><\/article><\/li><li class=\"wp-block-latest-comments__comment\"><img alt='' src='https:\/\/secure.gravatar.com\/avatar\/26fe54b5fa98d22d57e810a86c78d72eb27311b614566ccf5571cf4cb3863a41?s=48&#038;d=mm&#038;r=g' srcset='https:\/\/secure.gravatar.com\/avatar\/26fe54b5fa98d22d57e810a86c78d72eb27311b614566ccf5571cf4cb3863a41?s=96&#038;d=mm&#038;r=g 2x' class='avatar avatar-48 photo wp-block-latest-comments__comment-avatar' height='48' width='48' \/><article><footer class=\"wp-block-latest-comments__comment-meta\"><a class=\"wp-block-latest-comments__comment-author\" href=\"https:\/\/perlhacks.com\/\">Dave Cross<\/a> on <a class=\"wp-block-latest-comments__comment-link\" href=\"https:\/\/www.aibistin.com\/?p=23#comment-4423\">Upload and Email a photo using Dancer2 with Twitter Bootstrap<\/a><time datetime=\"2017-03-12T04:00:46-04:00\" class=\"wp-block-latest-comments__comment-date\">March 12, 2017<\/time><\/footer><div class=\"wp-block-latest-comments__comment-excerpt\"><p>Support for the Template Toolkit is already baked into Dancer2. The &#8220;template&#8221; function that you call is a Dancer2 function,&hellip;<\/p>\n<\/div><\/article><\/li><\/ol>","protected":false},"excerpt":{"rendered":"<p>I found myself needing some New York City detailed Zip Code information for another script I was creating. The zip codes themselves are easy enough to find online. I needed to include more details about each zip code location.&nbsp; I created a Perl script to merge two hard coded Perl data structures, which are printed [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[104,106,107,41,103,101],"tags":[47,48,29,35,49,30,20,96,50],"class_list":["post-673","post","type-post","status-publish","format-standard","hentry","category-data","category-database","category-moo","category-mooxoptions-perl","category-new-york-city","category-perl","tag-fileserialize","tag-json","tag-moo","tag-mooxoptions","tag-newyorkcity","tag-pathtiny","tag-perl","tag-zip-codes","tag-zipcodes"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.aibistin.com\/index.php?rest_route=\/wp\/v2\/posts\/673","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.aibistin.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aibistin.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aibistin.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aibistin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=673"}],"version-history":[{"count":21,"href":"https:\/\/www.aibistin.com\/index.php?rest_route=\/wp\/v2\/posts\/673\/revisions"}],"predecessor-version":[{"id":726,"href":"https:\/\/www.aibistin.com\/index.php?rest_route=\/wp\/v2\/posts\/673\/revisions\/726"}],"wp:attachment":[{"href":"https:\/\/www.aibistin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=673"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aibistin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=673"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aibistin.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=673"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}