Based on the two brothers’ writing methods
@agree and accept @xiaojieluoff

If the length of the number is fixed to the first six digits, the fastest way to write it is the first one below

import time lst = ['000001_2017-03-17.csv', '000001_2017-03-20.csv', '000002_2017-03-21.csv', '000002_2017-03-22.csv', '000003_2017-03-23.csv', '000004_2017-03-24.csv'] * 1000000 start = time.time() data = {_[:6] for _ in lst} print 'dic: {}'.format(time.time() - start) start = time.time() data = set(_[:6] for _ in lst) print 'set: {}'.format(time.time() - start) start = time.time() data = set(map(lambda _: _[:6], lst)) print('map：{}'.format(time.time() - start)) start = time.time() data = set() [data.add(_[:6]) for _ in lst] print('for：{}'.format(time.time() - start)) 耗时： dic: 0.72798705101 set: 0.929664850235 map：1.89214396477 for：1.76194214821

Use map and anonymous functions

lists = ['000001_2017-03-17.csv', '000001_2017-03-20.csv','000002_2017-03-21.csv','000002_2017-03-22.csv','000003_2017-03-23.csv', '000004_2017-03-24.csv'] data = list(set(map(lambda x:x.split('_')[0], lists))) print(data)

Output:

['000003', '000004', '000001', '000002']

Run the following code and you can see that with 6 million pieces of data, map is about 0.6s faster than for

import time lists = ['000001_2017-03-17.csv', '000001_2017-03-20.csv', '000002_2017-03-21.csv', '000002_2017-03-22.csv', '000003_2017-03-23.csv', '000004_2017-03-24.csv'] * 1000000 map_start = time.clock() map_data = list(set(map(lambda x:x.split('_')[0], lists))) map_end = (time.clock() - map_start) print('map 运行时间：{}'.format(map_end)) for_start = time.clock() data = set() for k in lists: data.add(k.split('_')[0]) for_end = (time.clock() - for_start) print('for 运行时间：{}'.format(for_end))

Output:

map 运行时间：2.36173 for 运行时间：2.9405870000000003

If the test data is expanded to 60 million, the gap will be even more obvious

map 运行时间：29.620203 for 运行时间：33.132621