Refactor events collection?

[eyalcohen]

Hey @sanderpick,

started to look at creating some of the DB tools. Taking merge crags as an example, most of the work is pretty straightforward. Updating the events collection will be the most complicated, because we have duplication of data there. Here is an example session event.

> db.event.find({action_type: 'session'})[0]
{
    "_id" : ObjectId("5487836f8d3ed78c37af88af"),
    "actor_id" : ObjectId("4ff2069f09dfc99a68000305"),
    "target_id" : ObjectId("516cf537bdc0f71fdd0074b9"),
    "action_id" : ObjectId("5487836f8d3ed78c37af88aa"),
    "action_type" : "session",
    "data" : {
        "action" : {
            "i" : "4ff2069f09dfc99a68000305",
            "a" : "Ryan Holm",
            "g" : "9215b51dc90fcfb197260767f6f7d8b7",
            "t" : "session",
            "b" : "",
            "n" : "12.09.14",
            "d" : "2014-12-09T23:19:10.896Z",
            "s" : "sessions/478597960"
        },
        "target" : {
            "s" : "crags/usa/whiskey-gulch",
            "l" : {
                "latitude" : 45.88248022935046,
                "longitude" : -112.30971336364746
            },
            "n" : "Whiskey Gulch",
            "c" : "United States"
        }
    },
    "date" : ISODate("2014-12-09T23:19:10.896Z"),
    "created" : ISODate("2014-12-09T23:19:11.417Z"),
    "updated" : ISODate("2014-12-09T23:19:11.417Z")
}

and the target_id inflated:

> db.crag.find({_id: ObjectId("516cf537bdc0f71fdd0074b9")})[0]
{
    "_id" : ObjectId("516cf537bdc0f71fdd0074b9"),
    "bcnt" : 243,
    "bgrdl" : "4",
    "bgrdu" : "7c+",
    "city" : null,
    "country" : "United States",
    "country_id" : ObjectId("516ce97fbdc0f71fdd000003"),
    "created" : ISODate("2013-04-16T09:49:54.913Z"),
    "key" : "usa/whiskey-gulch",
    "location" : {
        "latitude" : 45.88248022935046,
        "longitude" : -112.30971336364746
    },
    "name" : "Whiskey Gulch",
    "rcnt" : 0,
    "rgrdl" : "9c",
    "rgrdu" : "3",
    "updated" : ISODate("2013-07-30T00:02:09.429Z")
}

Notice that data.target on the event duplicates quite a bit of information.

Wondering if it would be simpler to refactor Events in the database without the data property, and then fill them as necessary. Of course, the downside is that you increased the number of hits to the database and probably slow down the whole operation.

Whatcha think?

[sanderpick]

Yeah, we can def do some improvements there. The original intention of the
"data" attribute was to make sending a notification very simple - you just
need to grab the event and you have all that you need for an email, sms,
in-browser, etc. This is pretty ingrained in the site at the moment...
however, I guess it will just become harder and harder to change.

im out the door for buddy's bachelor party weekend. we're gonna backpack
the lost coast! trust your judgement 100% on this. will be back on sunday
night.

On Thu, Aug 13, 2015 at 9:31 AM, Eyal Cohen [email protected]
wrote:

Hey @sanderpick https://github.com/sanderpick,

started to look at creating some of the DB tools. Taking merge crags as
an example, most of the work is pretty straightforward. Updating the events
collection will be the most complicated, because we have duplication of
data there. Here is an example session event.

db.event.find({action_type: 'session'})[0]
{
"_id" : ObjectId("5487836f8d3ed78c37af88af"),
"actor_id" : ObjectId("4ff2069f09dfc99a68000305"),
"target_id" : ObjectId("516cf537bdc0f71fdd0074b9"),
"action_id" : ObjectId("5487836f8d3ed78c37af88aa"),
"action_type" : "session",
"data" : {
"action" : {
"i" : "4ff2069f09dfc99a68000305",
"a" : "Ryan Holm",
"g" : "9215b51dc90fcfb197260767f6f7d8b7",
"t" : "session",
"b" : "",
"n" : "12.09.14",
"d" : "2014-12-09T23:19:10.896Z",
"s" : "sessions/478597960"
},
"target" : {
"s" : "crags/usa/whiskey-gulch",
"l" : {
"latitude" : 45.88248022935046,
"longitude" : -112.30971336364746
},
"n" : "Whiskey Gulch",
"c" : "United States"
}
},
"date" : ISODate("2014-12-09T23:19:10.896Z"),
"created" : ISODate("2014-12-09T23:19:11.417Z"),
"updated" : ISODate("2014-12-09T23:19:11.417Z")
}

and the target_id inflated:

db.crag.find({_id: ObjectId("516cf537bdc0f71fdd0074b9")})[0]
{
"_id" : ObjectId("516cf537bdc0f71fdd0074b9"),
"bcnt" : 243,
"bgrdl" : "4",
"bgrdu" : "7c+",
"city" : null,
"country" : "United States",
"country_id" : ObjectId("516ce97fbdc0f71fdd000003"),
"created" : ISODate("2013-04-16T09:49:54.913Z"),
"key" : "usa/whiskey-gulch",
"location" : {
"latitude" : 45.88248022935046,
"longitude" : -112.30971336364746
},
"name" : "Whiskey Gulch",
"rcnt" : 0,
"rgrdl" : "9c",
"rgrdu" : "3",
"updated" : ISODate("2013-07-30T00:02:09.429Z")
}

Notice that data.target on the event duplicates quite a bit of
information.

Wondering if it would be simpler to refactor Events in the database
without the data property, and then fill them as necessary. Of course,
the downside is that you increased the number of hits to the database and
probably slow down the whole operation.

Whatcha think?

—
Reply to this email directly or view it on GitHub
#135.

[eyalcohen]

It would be a lot of work, and would rip the site up. Not sure if its worth it or not...

Off to the Emeralds myself for the weekend, have fun!

[eyalcohen]

Think about it, maybe there's a better way thats less intrusive.

[eyalcohen]

I was playing with @ mentions today, and I still think something like this would be beneficial to do before we add much more to the site. Hard to balance infrastructure stuff with getting more features out though!

Probably white board sesh worthy. I need a diagram of how events/subs/notifications work because I keep forgetting =)

[sanderpick]

yeah I hear ya - more and more the duplicated info under "data" is not even
used... getting pushed out naturally which is a good sign to remove it! I
don't think it would be a huge effort

On Wed, Sep 23, 2015 at 1:39 PM, Eyal Cohen [email protected]
wrote:

I was playing with @mentions https://github.com/blog/821 today, and I
still think something like this would be beneficial to do before we add
much more to the site. Hard to balance infrastructure stuff with getting
more features out though!

Probably white board sesh worthy. I need a diagram of how
events/subs/notifications work because I keep forgetting =)

—
Reply to this email directly or view it on GitHub
#135 (comment)
.