Writing a CSV splitter in Python

The Issue To Solve

Lately in my coding journey I’ve been writing tools that help solve small productivity issues. This week I was talking with someone who had an issue related to splitting up data into multiple spreadsheets, so I decided to write a tool to help them complete that task.

Creating The CSV Splitter

Below I’ll outline the steps I took to create the tool, explaining each function and then showing the final script.

Main()

The main function does a few things. First it takes in input from the user to grab the location of the original CSV file and then asks how many rows each split CSV file should contain.

import csv...def main():    csv_file_path = input('Enter csv file path: ')
split_threshold = int(input('Enter how many rows per CSV...'))
with open(csv_file_path) as csv_file:
csv_reader = csv.DictReader(csv_file)
column_headers = csv_reader.fieldnames
loop_through_csv(
csv_reader,
split_threshold,
column_headers
)
if __name__ == "__main__":
main()

write_new_csv()

The write_new_csv() function takes in a csv file that is currently open and being written, the names of the column headers in the original csv (stored as an array of strings), and the split_number which assigns a number to each CSV file as they are split and saved. The names would end up being for example: CSV_1, CSV_2, CSV_3, etc.

def write_new_csv(csv_file_write, column_headers, split_number):
writer = csv.DictWriter(csv_file_write, fieldnames=column_headers)
writer.writeheader()
return writer

loop_through_csv()

This is where the important logic of the script lives. It’s a bit longer of a function so I’ll show it here and explain below.

def loop_through_csv(csv, split_threshold, column_headers):    row_counter = 0
split_number = 0
write_csv = None
for row in csv: if (row_counter == 0) or (row_counter == split_threshold):
if write_csv:
write_csv.close()
split_number += 1
write_csv = open(f'CSV_{split_number}.csv', 'w')
writer = write_new_csv(
write_csv,
column_headers,
split_number
)
writer.writerow(row)
row_counter = 0
else:
writer.writerow(row)
row_counter += 1
writer.writerow({'first_name': 'First', 'last_name': 'Last'})

Full Script

Full CSV Splitter Script

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Arthur Wilton

Arthur Wilton

41 Followers

Software Developer and Video/Post Production Professional. Recent graduate of Flatiron School.