Analyzing distances between clusters of orders











up vote
0
down vote

favorite












I wrote the Python class below, which does what I want it to do, but the data structure is a mess. Was wondering if there was a better structure I could use to get the same results but with better readable code.



Idea here is we retrieve a dataset from SQL(constructor), cluster the dataset into distinct keys(constructor), iterate through the keys and isolate the matching criteria in the dataset(organizer), pass those data chunks to map_loc_to_lat_long which will find all possible combinations of rows in the chunk and find the straight line distance between all the combination Lat Longs.



class OrderProximityModel:
def __init__(self, date):
self.date = str(date)
self.order_data = OrderProxDao().Load_Order_Lat_Long_Date_Zone_Data(self.date)
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])


def organizer(self):
container =
for date_zone in self.distinct:
latlng = list(filter(lambda x: str(x.Requirements) + ' ' + str(x.Route_Date) == date_zone, self.order_data))
for i in self.map_loc_to_lat_long(latlng):
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
InsertHelpers(container).chunk_maker(100)
return True


def map_loc_to_lat_long(self, grouped_zone):
converted = {}
for row in grouped_zone:
converted[row.LocationKey] = [row.Latitude, row.Longitude, row.DA, row.Route_Date, row.Requirements, row.DA]
grouped_combos = self.combo_creator(converted.keys())
return map(lambda combo: ([converted[combo[0]][2:] + [combo[0]] + [combo[1]] +
[StraightLineDistance().dist_cal(converted[combo[0]][0],
converted[combo[0]][1],
converted[combo[1]][0],
converted[combo[1]][1])]],
), grouped_combos)

@staticmethod
def combo_creator(inputs):
out =
for index, value in enumerate(inputs):
for nex_value in inputs[index + 1:]:
out.append((value, nex_value))
return out









share|improve this question









New contributor




Mike Sivalls is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















  • What are OrderProxDao, InsertHelpers, StraightLineDistance?
    – Graipher
    yesterday















up vote
0
down vote

favorite












I wrote the Python class below, which does what I want it to do, but the data structure is a mess. Was wondering if there was a better structure I could use to get the same results but with better readable code.



Idea here is we retrieve a dataset from SQL(constructor), cluster the dataset into distinct keys(constructor), iterate through the keys and isolate the matching criteria in the dataset(organizer), pass those data chunks to map_loc_to_lat_long which will find all possible combinations of rows in the chunk and find the straight line distance between all the combination Lat Longs.



class OrderProximityModel:
def __init__(self, date):
self.date = str(date)
self.order_data = OrderProxDao().Load_Order_Lat_Long_Date_Zone_Data(self.date)
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])


def organizer(self):
container =
for date_zone in self.distinct:
latlng = list(filter(lambda x: str(x.Requirements) + ' ' + str(x.Route_Date) == date_zone, self.order_data))
for i in self.map_loc_to_lat_long(latlng):
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
InsertHelpers(container).chunk_maker(100)
return True


def map_loc_to_lat_long(self, grouped_zone):
converted = {}
for row in grouped_zone:
converted[row.LocationKey] = [row.Latitude, row.Longitude, row.DA, row.Route_Date, row.Requirements, row.DA]
grouped_combos = self.combo_creator(converted.keys())
return map(lambda combo: ([converted[combo[0]][2:] + [combo[0]] + [combo[1]] +
[StraightLineDistance().dist_cal(converted[combo[0]][0],
converted[combo[0]][1],
converted[combo[1]][0],
converted[combo[1]][1])]],
), grouped_combos)

@staticmethod
def combo_creator(inputs):
out =
for index, value in enumerate(inputs):
for nex_value in inputs[index + 1:]:
out.append((value, nex_value))
return out









share|improve this question









New contributor




Mike Sivalls is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















  • What are OrderProxDao, InsertHelpers, StraightLineDistance?
    – Graipher
    yesterday













up vote
0
down vote

favorite









up vote
0
down vote

favorite











I wrote the Python class below, which does what I want it to do, but the data structure is a mess. Was wondering if there was a better structure I could use to get the same results but with better readable code.



Idea here is we retrieve a dataset from SQL(constructor), cluster the dataset into distinct keys(constructor), iterate through the keys and isolate the matching criteria in the dataset(organizer), pass those data chunks to map_loc_to_lat_long which will find all possible combinations of rows in the chunk and find the straight line distance between all the combination Lat Longs.



class OrderProximityModel:
def __init__(self, date):
self.date = str(date)
self.order_data = OrderProxDao().Load_Order_Lat_Long_Date_Zone_Data(self.date)
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])


def organizer(self):
container =
for date_zone in self.distinct:
latlng = list(filter(lambda x: str(x.Requirements) + ' ' + str(x.Route_Date) == date_zone, self.order_data))
for i in self.map_loc_to_lat_long(latlng):
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
InsertHelpers(container).chunk_maker(100)
return True


def map_loc_to_lat_long(self, grouped_zone):
converted = {}
for row in grouped_zone:
converted[row.LocationKey] = [row.Latitude, row.Longitude, row.DA, row.Route_Date, row.Requirements, row.DA]
grouped_combos = self.combo_creator(converted.keys())
return map(lambda combo: ([converted[combo[0]][2:] + [combo[0]] + [combo[1]] +
[StraightLineDistance().dist_cal(converted[combo[0]][0],
converted[combo[0]][1],
converted[combo[1]][0],
converted[combo[1]][1])]],
), grouped_combos)

@staticmethod
def combo_creator(inputs):
out =
for index, value in enumerate(inputs):
for nex_value in inputs[index + 1:]:
out.append((value, nex_value))
return out









share|improve this question









New contributor




Mike Sivalls is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











I wrote the Python class below, which does what I want it to do, but the data structure is a mess. Was wondering if there was a better structure I could use to get the same results but with better readable code.



Idea here is we retrieve a dataset from SQL(constructor), cluster the dataset into distinct keys(constructor), iterate through the keys and isolate the matching criteria in the dataset(organizer), pass those data chunks to map_loc_to_lat_long which will find all possible combinations of rows in the chunk and find the straight line distance between all the combination Lat Longs.



class OrderProximityModel:
def __init__(self, date):
self.date = str(date)
self.order_data = OrderProxDao().Load_Order_Lat_Long_Date_Zone_Data(self.date)
self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])


def organizer(self):
container =
for date_zone in self.distinct:
latlng = list(filter(lambda x: str(x.Requirements) + ' ' + str(x.Route_Date) == date_zone, self.order_data))
for i in self.map_loc_to_lat_long(latlng):
container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))
InsertHelpers(container).chunk_maker(100)
return True


def map_loc_to_lat_long(self, grouped_zone):
converted = {}
for row in grouped_zone:
converted[row.LocationKey] = [row.Latitude, row.Longitude, row.DA, row.Route_Date, row.Requirements, row.DA]
grouped_combos = self.combo_creator(converted.keys())
return map(lambda combo: ([converted[combo[0]][2:] + [combo[0]] + [combo[1]] +
[StraightLineDistance().dist_cal(converted[combo[0]][0],
converted[combo[0]][1],
converted[combo[1]][0],
converted[combo[1]][1])]],
), grouped_combos)

@staticmethod
def combo_creator(inputs):
out =
for index, value in enumerate(inputs):
for nex_value in inputs[index + 1:]:
out.append((value, nex_value))
return out






python python-2.x clustering geospatial






share|improve this question









New contributor




Mike Sivalls is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|improve this question









New contributor




Mike Sivalls is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this question




share|improve this question








edited 2 days ago









Mast

7,43863686




7,43863686






New contributor




Mike Sivalls is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked Nov 22 at 20:10









Mike Sivalls

61




61




New contributor




Mike Sivalls is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Mike Sivalls is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Mike Sivalls is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.












  • What are OrderProxDao, InsertHelpers, StraightLineDistance?
    – Graipher
    yesterday


















  • What are OrderProxDao, InsertHelpers, StraightLineDistance?
    – Graipher
    yesterday
















What are OrderProxDao, InsertHelpers, StraightLineDistance?
– Graipher
yesterday




What are OrderProxDao, InsertHelpers, StraightLineDistance?
– Graipher
yesterday










1 Answer
1






active

oldest

votes

















up vote
1
down vote













self.date = str(date)


This is a pet peeve of mine. Stringly-typed variables are usually a bad idea. If you receive a datetime object, you should usually keep it as datetime until you actually need it to be a string.



Load_Order_Lat_Long_Date_Zone_Data


If at all possible, shorten this method name. Also, methods are lowercase by convention.



self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])


Here you make a generator, construct a list and then convert it to a set. Skip the list - the set constructor can accept generators directly. Better yet, if you're in a sane version of Python, just use a set literal (and use a format string):



self.distinct = {'%s %s' % (row.Requirements, row.Route_Date) for row in self.order_data}


Your container = / container.append() loop can be replaced by proper use of a generator. Same with out.



latlng does not (and should not) be materialized to a list. It should be left as a generator, since you only iterate over it once.



This:



container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))


can be:



container.append(tuple(i[0][0][j] for j in (0, 1, 2, 4, 5, 6)))





share|improve this answer





















    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "196"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });






    Mike Sivalls is a new contributor. Be nice, and check out our Code of Conduct.










     

    draft saved


    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f208247%2fanalyzing-distances-between-clusters-of-orders%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    1
    down vote













    self.date = str(date)


    This is a pet peeve of mine. Stringly-typed variables are usually a bad idea. If you receive a datetime object, you should usually keep it as datetime until you actually need it to be a string.



    Load_Order_Lat_Long_Date_Zone_Data


    If at all possible, shorten this method name. Also, methods are lowercase by convention.



    self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])


    Here you make a generator, construct a list and then convert it to a set. Skip the list - the set constructor can accept generators directly. Better yet, if you're in a sane version of Python, just use a set literal (and use a format string):



    self.distinct = {'%s %s' % (row.Requirements, row.Route_Date) for row in self.order_data}


    Your container = / container.append() loop can be replaced by proper use of a generator. Same with out.



    latlng does not (and should not) be materialized to a list. It should be left as a generator, since you only iterate over it once.



    This:



    container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))


    can be:



    container.append(tuple(i[0][0][j] for j in (0, 1, 2, 4, 5, 6)))





    share|improve this answer

























      up vote
      1
      down vote













      self.date = str(date)


      This is a pet peeve of mine. Stringly-typed variables are usually a bad idea. If you receive a datetime object, you should usually keep it as datetime until you actually need it to be a string.



      Load_Order_Lat_Long_Date_Zone_Data


      If at all possible, shorten this method name. Also, methods are lowercase by convention.



      self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])


      Here you make a generator, construct a list and then convert it to a set. Skip the list - the set constructor can accept generators directly. Better yet, if you're in a sane version of Python, just use a set literal (and use a format string):



      self.distinct = {'%s %s' % (row.Requirements, row.Route_Date) for row in self.order_data}


      Your container = / container.append() loop can be replaced by proper use of a generator. Same with out.



      latlng does not (and should not) be materialized to a list. It should be left as a generator, since you only iterate over it once.



      This:



      container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))


      can be:



      container.append(tuple(i[0][0][j] for j in (0, 1, 2, 4, 5, 6)))





      share|improve this answer























        up vote
        1
        down vote










        up vote
        1
        down vote









        self.date = str(date)


        This is a pet peeve of mine. Stringly-typed variables are usually a bad idea. If you receive a datetime object, you should usually keep it as datetime until you actually need it to be a string.



        Load_Order_Lat_Long_Date_Zone_Data


        If at all possible, shorten this method name. Also, methods are lowercase by convention.



        self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])


        Here you make a generator, construct a list and then convert it to a set. Skip the list - the set constructor can accept generators directly. Better yet, if you're in a sane version of Python, just use a set literal (and use a format string):



        self.distinct = {'%s %s' % (row.Requirements, row.Route_Date) for row in self.order_data}


        Your container = / container.append() loop can be replaced by proper use of a generator. Same with out.



        latlng does not (and should not) be materialized to a list. It should be left as a generator, since you only iterate over it once.



        This:



        container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))


        can be:



        container.append(tuple(i[0][0][j] for j in (0, 1, 2, 4, 5, 6)))





        share|improve this answer












        self.date = str(date)


        This is a pet peeve of mine. Stringly-typed variables are usually a bad idea. If you receive a datetime object, you should usually keep it as datetime until you actually need it to be a string.



        Load_Order_Lat_Long_Date_Zone_Data


        If at all possible, shorten this method name. Also, methods are lowercase by convention.



        self.distinct = set([str(row.Requirements) + ' ' + str(row.Route_Date) for row in self.order_data])


        Here you make a generator, construct a list and then convert it to a set. Skip the list - the set constructor can accept generators directly. Better yet, if you're in a sane version of Python, just use a set literal (and use a format string):



        self.distinct = {'%s %s' % (row.Requirements, row.Route_Date) for row in self.order_data}


        Your container = / container.append() loop can be replaced by proper use of a generator. Same with out.



        latlng does not (and should not) be materialized to a list. It should be left as a generator, since you only iterate over it once.



        This:



        container.append((i[0][0][0], i[0][0][1], i[0][0][2], i[0][0][4], i[0][0][5], i[0][0][6]))


        can be:



        container.append(tuple(i[0][0][j] for j in (0, 1, 2, 4, 5, 6)))






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 23 at 5:19









        Reinderien

        1,392516




        1,392516






















            Mike Sivalls is a new contributor. Be nice, and check out our Code of Conduct.










             

            draft saved


            draft discarded


















            Mike Sivalls is a new contributor. Be nice, and check out our Code of Conduct.













            Mike Sivalls is a new contributor. Be nice, and check out our Code of Conduct.












            Mike Sivalls is a new contributor. Be nice, and check out our Code of Conduct.















             


            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f208247%2fanalyzing-distances-between-clusters-of-orders%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Morgemoulin

            Scott Moir

            Souastre