May 25, 2024
Note: This article is translated from my Japanese article.
This article first explains points to watch out for when analyzing municipal data, and then introduces how to smartly format municipal data in R using the jpcity package.
What should you be careful about when analyzing statistical data broken down by municipality? Below, I’ll introduce three main points to watch out for.
The first thing to watch out for is that there can be multiple municipalities with the same name. In fact, as of January 1, 2020, there are just under 60 municipalities whose kanji notation overlaps with another, as shown below1.
Here, the 5-digit number in the leftmost column of the table, city_code, is called the municipality code (the National Local Government Code), a number used to uniquely identify each municipality. For this reason, when analyzing municipal data, it’s preferable to identify municipalities by their municipality code rather than by name.
Incidentally, the first two digits of the municipality code are the identification number for the prefecture, known as the prefecture code.
Does that mean everything is fine as long as you use municipality codes? Unfortunately, there is another pitfall in analysis based on municipality codes: municipality codes can change due to municipal mergers (haichi-bungō, i.e., the abolition, establishment, division, and merger of municipalities).
Municipality codes came into use on April 1, 1970, but the number of municipalities subsequently decreased significantly due to the Great Heisei Mergers (1999-2010) and other consolidations. Because of this, analyzing municipal data from around 1999-2010 in particular can require separate, tedious aggregation and apportionment processing (so-called haichi-bungō processing).
As an example, let’s extract the municipality formed from the largest number of merged municipalities between April 1, 1970 and January 1, 2020. The results (table below) show that the municipality that has undergone the most municipal mergers is Joetsu City in Niigata Prefecture, formed from the merger of 15 municipalities.
Municipality codes can also change due to reasons other than mergers, such as a village being reclassified as a town or a town being reclassified as a city. For this reason, when analyzing municipal data, it can be necessary to format the data while taking into account the reason for and timing of these municipality code changes.
tibble(city_after = get_city("2020-01-01"),
city_before = city_after |>
city_convert("2020-01-01", "1970-04-01")) |>
mutate(size_city_before = vctrs::list_sizes(city_before)) |>
slice_max(size_city_before,
n = 1) |>
select(!size_city_before) |>
unnest(city_before) |>
mutate(city_code_before = city_code(city_before),
city_name_before = city_name(city_before),
city_code_after = city_code(city_after),
city_name_after = city_name(city_after),
.keep = "unused") |>
rmarkdown::paged_table()Once you’ve grasped the points above, you can say you’ve cleared the biggest hurdle in formatting municipal data. Finally, though it’s a minor point, let’s look at how wards of designated cities are handled.
Depending on the statistical data, you may only be able to obtain municipal data aggregated at the city level, rather than municipal data available at the ward level for designated cities.
Since the city and its wards in a designated city are each assigned different municipality codes, mapping between municipality codes is necessary when integrating data with these different regional breakdowns.
As an example, the table below shows the mapping between the city and its wards for Yokohama City.
tibble(city = find_city("横浜市", "2020-01-01") |>
city_desig_merge() |>
vctrs::vec_unique(),
city_desig = city |>
city_desig_split()) |>
unnest(city_desig) |>
mutate(city_code = city_code(city),
city_name = city_name(city),
city_desig_code = city_code(city_desig),
city_desig_name = city_name(city_desig),
.keep = "unused") |>
rmarkdown::paged_table()With the points above in mind, let’s introduce how to smartly format municipal data using R. In fact, all of the examples shown above are based on information published by e-Stat in “Search Municipalities” and “Search Municipal Merger Information”.
However, “Search Municipal Merger Information” describes changes to municipality codes due to mergers and other events in text form, which makes it difficult to use directly for data formatting.
So, to make it easy to perform haichi-bungō processing and map between the cities and wards of designated cities without having to do this kind of complex data formatting, I created a new package called jpcity2. Incidentally, all of the tables shown above were created using the jpcity package. Since jpcity is registered on CRAN, it can be installed by running the code below. From here, I’ll briefly explain how to use jpcity.
In jpcity, you can build a municipality object from a municipality code using the parse_city() function3. You can also build a municipality object from a municipality name using the find_city() function. However, since find_city() returns all matching municipalities when the name is a duplicate, it’s generally recommended to use parse_city(). Furthermore, both parse_city() and find_city() let you specify a date as the second argument, when, to retrieve the municipality as of that point in time4.
You can also use the city_code() and city_name() functions to convert a municipality object to a (character) municipality code or municipality name, respectively.
You can perform municipal merger (haichi-bungō) processing using the city_convert() function. Here, let’s take the Joetsu City we read in earlier as an example and convert it to the municipality as of January 1, 20005. For city_convert(), specify the pre-conversion date "2020-01-01" as the second argument, from, and the post-conversion date "2000-01-01" as the third argument, to.
city |>
select(city_parsed) |>
mutate(city_converted = city_parsed |>
city_convert("2020-01-01", "2000-01-01")) |>
unnest(city_converted)# A tibble: 14 × 2
city_parsed city_converted
<city> <city>
1 15222 [新潟県上越市] 15521 [新潟県安塚町]
2 15222 [新潟県上越市] 15522 [新潟県浦川原村]
3 15222 [新潟県上越市] 15525 [新潟県大島村]
4 15222 [新潟県上越市] 15526 [新潟県牧村]
5 15222 [新潟県上越市] 15541 [新潟県柿崎町]
6 15222 [新潟県上越市] 15542 [新潟県大潟町]
7 15222 [新潟県上越市] 15543 [新潟県頸城村]
8 15222 [新潟県上越市] 15544 [新潟県吉川町]
9 15222 [新潟県上越市] 15546 [新潟県中郷村]
10 15222 [新潟県上越市] 15548 [新潟県板倉町]
11 15222 [新潟県上越市] 15549 [新潟県清里村]
12 15222 [新潟県上越市] 15550 [新潟県三和村]
13 15222 [新潟県上越市] 15561 [新潟県名立町]
14 15222 [新潟県上越市] 15222 [新潟県上越市] You can map between the city and wards of a designated city using the city_desig_split() and city_desig_merge() functions. Here, let’s perform this mapping for Yokohama City, the example we used earlier.
The city_desig_split() function splits a designated city into its wards. Using city_desig_split(), we can see that Yokohama City has 14 wards. The city_desig_merge() function merges the wards of a designated city back into the city. Using city_desig_merge(), we can convert the wards of Yokohama City we just split back into the city.
# Split the designated city into its wards
city_desig <- tibble(city = parse_city("14100", "2020-01-01"),
city_desig = city_desig_split(city)) |>
unnest(city_desig)
city_desig# A tibble: 14 × 2
city city_desig
<city> <city>
1 14100 [神奈川県横浜市] 14101 [神奈川県横浜市鶴見区]
2 14100 [神奈川県横浜市] 14102 [神奈川県横浜市神奈川区]
3 14100 [神奈川県横浜市] 14103 [神奈川県横浜市西区]
4 14100 [神奈川県横浜市] 14104 [神奈川県横浜市中区]
5 14100 [神奈川県横浜市] 14105 [神奈川県横浜市南区]
6 14100 [神奈川県横浜市] 14106 [神奈川県横浜市保土ケ谷区]
7 14100 [神奈川県横浜市] 14107 [神奈川県横浜市磯子区]
8 14100 [神奈川県横浜市] 14108 [神奈川県横浜市金沢区]
9 14100 [神奈川県横浜市] 14109 [神奈川県横浜市港北区]
10 14100 [神奈川県横浜市] 14110 [神奈川県横浜市戸塚区]
11 14100 [神奈川県横浜市] 14111 [神奈川県横浜市港南区]
12 14100 [神奈川県横浜市] 14112 [神奈川県横浜市旭区]
13 14100 [神奈川県横浜市] 14113 [神奈川県横浜市緑区]
14 14100 [神奈川県横浜市] 14114 [神奈川県横浜市瀬谷区] # Merge the wards of the designated city back into the city (showing only the first few rows)
city_desig |>
select(city_desig) |>
mutate(city = city_desig_merge(city_desig)) |>
head()# A tibble: 6 × 2
city_desig city
<city> <city>
1 14101 [神奈川県横浜市鶴見区] 14100 [神奈川県横浜市]
2 14102 [神奈川県横浜市神奈川区] 14100 [神奈川県横浜市]
3 14103 [神奈川県横浜市西区] 14100 [神奈川県横浜市]
4 14104 [神奈川県横浜市中区] 14100 [神奈川県横浜市]
5 14105 [神奈川県横浜市南区] 14100 [神奈川県横浜市]
6 14106 [神奈川県横浜市保土ケ谷区] 14100 [神奈川県横浜市]In this article, I introduced points to watch out for when analyzing municipal data, and how to smartly format municipal data in R using the jpcity package. If you have any requests, bug reports, or questions about the jpcity package, feel free to reach out via GitHub or the comment section below.
Here, municipalities are counted as “duplicates” if their kanji notation is the same, even if the hiragana reading differs. Also, ward names of designated cities (seirei shitei toshi) are excluded, since there are many overlaps among them.↩︎
jpcity is built on data published by e-Stat in “Search Municipalities” and “Search Municipal Merger Information”.↩︎
parse_city() also supports 6-digit municipality codes that include a check digit.↩︎
If the when argument isn’t specified, jpcity infers which period the municipality belongs to and reads it in accordingly. If municipalities with inconsistent periods are included, a read error will occur.↩︎
Joetsu City was newly established on April 29, 1971 through the merger of Takada City and Naoetsu City, and later, on January 1, 2005, 13 towns and villages were merged into Joetsu City.↩︎