Analyzing distances between clusters of orders
up vote
0
down vote
favorite
I wrote the Python class below, which does what I want it to do, but the data structure is a mess. Was wondering if there was a better structure I could use to get the same results but with better readable code.
Idea here is we retrieve a dataset from SQL(constructor), cluster the dataset into distinct keys(constructor), iterate through the keys and isolate the matching criteria in the dataset(organizer), pass those data chunks to map_loc_to_lat_long which will find all possible combinations of rows in the chunk and find the straight line distance between all the combination Lat Longs.
class OrderProximityModel:
def __init__(self, date):
self.date = str(date)
self.order_data = OrderProxDao().Load_Order_Lat_Long_Date_Zone_Data(self.date)
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])
def organizer(self):
container =
for date_zone in self.distinct:
latlng = list(filter(lambda x: str(x.Requirements) + ' ' + str(x.Route_Date) == date_zone, self.order_data))
for i in self.map_loc_to_lat_long(latlng):
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
InsertHelpers(container).chunk_maker(100)
return True
def map_loc_to_lat_long(self, grouped_zone):
converted = {}
for row in grouped_zone:
converted[row.LocationKey] = [row.Latitude, row.Longitude, row.DA, row.Route_Date, row.Requirements, row.DA]
grouped_combos = self.combo_creator(converted.keys())
return map(lambda combo: ([converted[combo[0]][2:] + [combo[0]] + [combo[1]] +
[StraightLineDistance().dist_cal(converted[combo[0]][0],
converted[combo[0]][1],
converted[combo[1]][0],
converted[combo[1]][1])]],
), grouped_combos)
@staticmethod
def combo_creator(inputs):
out =
for index, value in enumerate(inputs):
for nex_value in inputs[index + 1:]:
out.append((value, nex_value))
return out
python python-2.x clustering geospatial
New contributor
add a comment |
up vote
0
down vote
favorite
I wrote the Python class below, which does what I want it to do, but the data structure is a mess. Was wondering if there was a better structure I could use to get the same results but with better readable code.
Idea here is we retrieve a dataset from SQL(constructor), cluster the dataset into distinct keys(constructor), iterate through the keys and isolate the matching criteria in the dataset(organizer), pass those data chunks to map_loc_to_lat_long which will find all possible combinations of rows in the chunk and find the straight line distance between all the combination Lat Longs.
class OrderProximityModel:
def __init__(self, date):
self.date = str(date)
self.order_data = OrderProxDao().Load_Order_Lat_Long_Date_Zone_Data(self.date)
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])
def organizer(self):
container =
for date_zone in self.distinct:
latlng = list(filter(lambda x: str(x.Requirements) + ' ' + str(x.Route_Date) == date_zone, self.order_data))
for i in self.map_loc_to_lat_long(latlng):
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
InsertHelpers(container).chunk_maker(100)
return True
def map_loc_to_lat_long(self, grouped_zone):
converted = {}
for row in grouped_zone:
converted[row.LocationKey] = [row.Latitude, row.Longitude, row.DA, row.Route_Date, row.Requirements, row.DA]
grouped_combos = self.combo_creator(converted.keys())
return map(lambda combo: ([converted[combo[0]][2:] + [combo[0]] + [combo[1]] +
[StraightLineDistance().dist_cal(converted[combo[0]][0],
converted[combo[0]][1],
converted[combo[1]][0],
converted[combo[1]][1])]],
), grouped_combos)
@staticmethod
def combo_creator(inputs):
out =
for index, value in enumerate(inputs):
for nex_value in inputs[index + 1:]:
out.append((value, nex_value))
return out
python python-2.x clustering geospatial
New contributor
What areOrderProxDao
,InsertHelpers
,StraightLineDistance
?
– Graipher
yesterday
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I wrote the Python class below, which does what I want it to do, but the data structure is a mess. Was wondering if there was a better structure I could use to get the same results but with better readable code.
Idea here is we retrieve a dataset from SQL(constructor), cluster the dataset into distinct keys(constructor), iterate through the keys and isolate the matching criteria in the dataset(organizer), pass those data chunks to map_loc_to_lat_long which will find all possible combinations of rows in the chunk and find the straight line distance between all the combination Lat Longs.
class OrderProximityModel:
def __init__(self, date):
self.date = str(date)
self.order_data = OrderProxDao().Load_Order_Lat_Long_Date_Zone_Data(self.date)
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])
def organizer(self):
container =
for date_zone in self.distinct:
latlng = list(filter(lambda x: str(x.Requirements) + ' ' + str(x.Route_Date) == date_zone, self.order_data))
for i in self.map_loc_to_lat_long(latlng):
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
InsertHelpers(container).chunk_maker(100)
return True
def map_loc_to_lat_long(self, grouped_zone):
converted = {}
for row in grouped_zone:
converted[row.LocationKey] = [row.Latitude, row.Longitude, row.DA, row.Route_Date, row.Requirements, row.DA]
grouped_combos = self.combo_creator(converted.keys())
return map(lambda combo: ([converted[combo[0]][2:] + [combo[0]] + [combo[1]] +
[StraightLineDistance().dist_cal(converted[combo[0]][0],
converted[combo[0]][1],
converted[combo[1]][0],
converted[combo[1]][1])]],
), grouped_combos)
@staticmethod
def combo_creator(inputs):
out =
for index, value in enumerate(inputs):
for nex_value in inputs[index + 1:]:
out.append((value, nex_value))
return out
python python-2.x clustering geospatial
New contributor
I wrote the Python class below, which does what I want it to do, but the data structure is a mess. Was wondering if there was a better structure I could use to get the same results but with better readable code.
Idea here is we retrieve a dataset from SQL(constructor), cluster the dataset into distinct keys(constructor), iterate through the keys and isolate the matching criteria in the dataset(organizer), pass those data chunks to map_loc_to_lat_long which will find all possible combinations of rows in the chunk and find the straight line distance between all the combination Lat Longs.
class OrderProximityModel:
def __init__(self, date):
self.date = str(date)
self.order_data = OrderProxDao().Load_Order_Lat_Long_Date_Zone_Data(self.date)
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])
def organizer(self):
container =
for date_zone in self.distinct:
latlng = list(filter(lambda x: str(x.Requirements) + ' ' + str(x.Route_Date) == date_zone, self.order_data))
for i in self.map_loc_to_lat_long(latlng):
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
InsertHelpers(container).chunk_maker(100)
return True
def map_loc_to_lat_long(self, grouped_zone):
converted = {}
for row in grouped_zone:
converted[row.LocationKey] = [row.Latitude, row.Longitude, row.DA, row.Route_Date, row.Requirements, row.DA]
grouped_combos = self.combo_creator(converted.keys())
return map(lambda combo: ([converted[combo[0]][2:] + [combo[0]] + [combo[1]] +
[StraightLineDistance().dist_cal(converted[combo[0]][0],
converted[combo[0]][1],
converted[combo[1]][0],
converted[combo[1]][1])]],
), grouped_combos)
@staticmethod
def combo_creator(inputs):
out =
for index, value in enumerate(inputs):
for nex_value in inputs[index + 1:]:
out.append((value, nex_value))
return out
python python-2.x clustering geospatial
python python-2.x clustering geospatial
New contributor
New contributor
edited 2 days ago
Mast
7,43863686
7,43863686
New contributor
asked Nov 22 at 20:10
Mike Sivalls
61
61
New contributor
New contributor
What areOrderProxDao
,InsertHelpers
,StraightLineDistance
?
– Graipher
yesterday
add a comment |
What areOrderProxDao
,InsertHelpers
,StraightLineDistance
?
– Graipher
yesterday
What are
OrderProxDao
, InsertHelpers
, StraightLineDistance
?– Graipher
yesterday
What are
OrderProxDao
, InsertHelpers
, StraightLineDistance
?– Graipher
yesterday
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
self.date = str(date)
This is a pet peeve of mine. Stringly-typed variables are usually a bad idea. If you receive a datetime object, you should usually keep it as datetime until you actually need it to be a string.
Load_Order_Lat_Long_Date_Zone_Data
If at all possible, shorten this method name. Also, methods are lowercase by convention.
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])
Here you make a generator, construct a list and then convert it to a set. Skip the list - the set
constructor can accept generators directly. Better yet, if you're in a sane version of Python, just use a set literal (and use a format string):
self.distinct = {'%s %s' % (row.Requirements, row.Route_Date) for row in self.order_data}
Your container =
/ container.append()
loop can be replaced by proper use of a generator. Same with out
.
latlng
does not (and should not) be materialized to a list. It should be left as a generator, since you only iterate over it once.
This:
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
can be:
container.append(tuple(i[0][0][j] for j in (0, 1, 2, 4, 5, 6)))
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
self.date = str(date)
This is a pet peeve of mine. Stringly-typed variables are usually a bad idea. If you receive a datetime object, you should usually keep it as datetime until you actually need it to be a string.
Load_Order_Lat_Long_Date_Zone_Data
If at all possible, shorten this method name. Also, methods are lowercase by convention.
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])
Here you make a generator, construct a list and then convert it to a set. Skip the list - the set
constructor can accept generators directly. Better yet, if you're in a sane version of Python, just use a set literal (and use a format string):
self.distinct = {'%s %s' % (row.Requirements, row.Route_Date) for row in self.order_data}
Your container =
/ container.append()
loop can be replaced by proper use of a generator. Same with out
.
latlng
does not (and should not) be materialized to a list. It should be left as a generator, since you only iterate over it once.
This:
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
can be:
container.append(tuple(i[0][0][j] for j in (0, 1, 2, 4, 5, 6)))
add a comment |
up vote
1
down vote
self.date = str(date)
This is a pet peeve of mine. Stringly-typed variables are usually a bad idea. If you receive a datetime object, you should usually keep it as datetime until you actually need it to be a string.
Load_Order_Lat_Long_Date_Zone_Data
If at all possible, shorten this method name. Also, methods are lowercase by convention.
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])
Here you make a generator, construct a list and then convert it to a set. Skip the list - the set
constructor can accept generators directly. Better yet, if you're in a sane version of Python, just use a set literal (and use a format string):
self.distinct = {'%s %s' % (row.Requirements, row.Route_Date) for row in self.order_data}
Your container =
/ container.append()
loop can be replaced by proper use of a generator. Same with out
.
latlng
does not (and should not) be materialized to a list. It should be left as a generator, since you only iterate over it once.
This:
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
can be:
container.append(tuple(i[0][0][j] for j in (0, 1, 2, 4, 5, 6)))
add a comment |
up vote
1
down vote
up vote
1
down vote
self.date = str(date)
This is a pet peeve of mine. Stringly-typed variables are usually a bad idea. If you receive a datetime object, you should usually keep it as datetime until you actually need it to be a string.
Load_Order_Lat_Long_Date_Zone_Data
If at all possible, shorten this method name. Also, methods are lowercase by convention.
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])
Here you make a generator, construct a list and then convert it to a set. Skip the list - the set
constructor can accept generators directly. Better yet, if you're in a sane version of Python, just use a set literal (and use a format string):
self.distinct = {'%s %s' % (row.Requirements, row.Route_Date) for row in self.order_data}
Your container =
/ container.append()
loop can be replaced by proper use of a generator. Same with out
.
latlng
does not (and should not) be materialized to a list. It should be left as a generator, since you only iterate over it once.
This:
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
can be:
container.append(tuple(i[0][0][j] for j in (0, 1, 2, 4, 5, 6)))
self.date = str(date)
This is a pet peeve of mine. Stringly-typed variables are usually a bad idea. If you receive a datetime object, you should usually keep it as datetime until you actually need it to be a string.
Load_Order_Lat_Long_Date_Zone_Data
If at all possible, shorten this method name. Also, methods are lowercase by convention.
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])
Here you make a generator, construct a list and then convert it to a set. Skip the list - the set
constructor can accept generators directly. Better yet, if you're in a sane version of Python, just use a set literal (and use a format string):
self.distinct = {'%s %s' % (row.Requirements, row.Route_Date) for row in self.order_data}
Your container =
/ container.append()
loop can be replaced by proper use of a generator. Same with out
.
latlng
does not (and should not) be materialized to a list. It should be left as a generator, since you only iterate over it once.
This:
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
can be:
container.append(tuple(i[0][0][j] for j in (0, 1, 2, 4, 5, 6)))
answered Nov 23 at 5:19
Reinderien
1,392516
1,392516
add a comment |
add a comment |
Mike Sivalls is a new contributor. Be nice, and check out our Code of Conduct.
Mike Sivalls is a new contributor. Be nice, and check out our Code of Conduct.
Mike Sivalls is a new contributor. Be nice, and check out our Code of Conduct.
Mike Sivalls is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f208247%2fanalyzing-distances-between-clusters-of-orders%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
What are
OrderProxDao
,InsertHelpers
,StraightLineDistance
?– Graipher
yesterday