I have a site where customers purchase items that are tagged with a variety of taxonomy terms. I want to create a group of customers who might be interested in the same items by considering the tags associated with purchases they've made. Rather than comparing a list of tags for each customer each time I want to build the group, I'm wondering if I can use some type of scoring to solve the problem.
The way I'm thinking about it, each tag would have some unique number assigned to it. When I perform a scoring operation it would render a number that could only be achieved by combining a specific set of tags.
I could update a customer's "score" periodically so that it remains relevant.
Am I on the right track? Any ideas?
Your description of the problem looks much more like a clustering or recommendation problem. I am not sure if those tags are enough of an information to use clustering or recommendation tough.
Your idea of the score doesn't look promising to me, because the same sum could be achieved in several ways, if those numbers aren't carefully enough chosen.
What I would suggest you:
You can store tags for each user. When some user purchases a new item, you will add the tags of the item to the user's tags. On periodical time you will update the users profiles. Let's say we have users A and B. If at the time of the update the similarity between A and B is greater than some threshold, you will add a relation between the users which will indicate that the two users are similar. If it's lower you will remove the relation (if previously they were related). The similarity could be either a number of common tags or num_common_tags / num_of_tags_assigned_either_in_A_or_B.
Later on, when you will want to get users with particular set of tags, you will just do a query which checks which users have that set of tags. Also you can check for similar users to given user, just by looking up which users are linked with the user in question.
If you assign a unique power of two to each tag, then you can sum the values corresponding to the tags, and users with the exact same sets of tags will get identical values.
red = 1
green = 2
blue = 4
yellow = 8
For example, only customers who have the set of { red, blue } will have a value of 5.
This is essentially using a bitmap to represent a set. The drawback is that if you have many tags, you'll quickly run out of integers. For example, if your (unsigned) integer type is four bytes, you'd be limited to 32 tags. There are libraries and classes that let you represent much larger bitsets, but, at that point, it's probably worth considering other approaches.
Another problem with this approach is that it doesn't help you cluster members that are similar but not identical.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
This is an algorithmic problem and I'm not sure it has a solution. I think it's a specific case of a more generic computer science problem that has no solution but I'd rather not disclose which one to avoid planting biases. It came up from a real life situation in which mobile phones were out of credit and thus, we didn't have long range communications.
Two groups of people, each with 2 people (but it might be true for N people) arranged to meet at the center of a park but at the time of meeting, the park is closed. Now, they'll have to meet somewhere else around the park. Is there an algorithm each and every single individual could follow to converge all in one point?
For example, if each group splits in two and goes around and when they find another person keep on going with that person, they would all converge on the other side of the park. But if the other group does the same, then, they wouldn't be able to take the found members of the other group with them. This is not a possible solution.
I'm not sure if I explained well enough. I can try to draw a diagram.
Deterministic Solution for N > 1, K > 1
For N groups of K people each.
Since the problem is based on people whose mobile phones are out of credit, let's assume that each person in each group has their own phone. If that's not acceptable, then substitute the phone with a credit card, social security, driver's license, or any other item with numerical identification that is guaranteed to be unique.
In each group, each person must remember the highest number among that group, and the person with the highest number (labeled leader) must travel clockwise around the perimeter while the rest of the group stays put.
After the leader of each group meets the next group, they compare their number with the group's previous leader number.
If the leader's number is higher than the group's previous leader's number, then the leader and the group all continue along the perimeter of the park. If the group's previous leader's number is higher, then they all stay put.
Eventually the leader with the highest number will continue around the entire perimeter exactly 1 rotation, collecting the entire group.
Deterministic solution for N > 1, K = 1 (with one reasonable assumption of knowledge ahead-of-time)
In this case, each group only contains one person. Let's assume that the number used is a phone number, because it is then reasonable to also assume that at least one pair of people will know each other's numbers and so one of them will stay put.
For N = 2, this becomes trivially reduced to one person staying put and the other person going around clockwise.
For other cases, the fact that at least two people will initially know each other's numbers will effectively increase the maximum K to at least 2 (because the person or people who stay put will continue to stay put if the person they know has a higher number than the leader who shows up to meet them), but we still have to introduce one more step to the algorithm to make sure it will terminate.
The extra step is that if a leader has continued around the perimeter for exactly one rotation without adding anyone to the group, then the leader must leave their group behind and start over for one more rotation around the perimeter. This means that a leader with no group will continue indefinitely until they find someone else, which is good.
With this extra step, it is easy to see why we have to assume that at least one pair of people need to know each other's phone numbers ahead of time, because then we can guarantee that the person who stays put will eventually accumulate the entire group.
Feel free to leave comments or suggestions to improve the algorithm I've laid out or challenge me if you think I missed an edge case. If not, then I hope you liked my answer.
Update
For fun, I decided to write a visual demo of my solutions to the problem using d3. Feel free to play around with the parameters and restart the simulation with any initial state. Here's the link:
https://jsfiddle.net/patrob10114/c3d478ty/show/
Key
black - leader
white - follower
when clicked
blue - selected person
green - known by selected person
red - unknown by selected person
Note that collaboration occurs at the start of every step, so if two groups just combined in the current step, most people won't know the people from the opposite group until after the next step is invoked.
They should move towards the northernmost point of the park.
I'd send both groups in a random direction. If they went a half circle without meeting the other group, rerandomize the directions. This will make them meet in a few rounds most of the time, however there is an infinitely small chance that they still never meet.
It is not possible with a deterministic algorithm if
• we have to meet at some point on the perimeter,
• we are unable to distinguish points on the perimeter (or the algorithm is not allowed to use such a distinction),
• we are unable to distinguish individuals in the groups (or the algorithm is not allowed to use such a distinction),
• the perimeter is circular (see below for a more general case),
• we all follow the same algorithm, and
• the initial points may be anywhere on the perimeter.
Proof: With a deterministic algorithm we can deduce the final positions from the initial positions, but the groups could start evenly spaced around the perimeter, in which case the problem has rotational symmetry and so the solution will be unchanged by a 1/n rotation, which however has no fixed point on the perimeter.
Status of assumptions
Dropping various assumptions leads, as others have observed to various solutions:
Non-deterministic: As others have observed, various non-deterministic algorithms do provide a solution whose probability of termination tends to certainty as time tends to infinity; I suspect almost any random walk would do. (Many answers)
Points indistinguishable: Agree on a fixed point at which to meet if needed: flyx’s answer.
Individuals indistinguishable: If there is a perfect hash algorithm, choose those with the lowest hash to collect others: Patrick Roberts’s solution.
Same algorithm: Choose one in advance to collect the others (adapting Patrick Roberts’s solution).
Other assumptions can be weakened:
Non-circular perimeter: The condition that the perimeter be circular is rather artificial, but if the perimeter is topologically equivalent to a circle, this equivalence can be used to convert any solution to a solution to the circle problem.
Unrestricted initial points: Even if the initial points cannot be evenly spaced, as long as some points are distinct, a topological equivalence (as for a non-circular perimeter) reduces a solution to a solution to the circular case, showing that no solution can exist.
I think this question really belongs on Computer Science Stack Exchange.
This question heavily depends on what kind of operations do we have and what do you consider your environment looks like. I asked your this questions with no reply, so here is my interpretation:
The park is a 2d space, 2 groups are located randomly, each group has the same right/left (both are facing the park). Both have the same operations are programmed to do absolutely the same things (nothing like I go right, and you go left, because this makes the problem obvious). So the operations are: Go right/left/stop for x units of time. They can also figure out that they passed through their original position (the one in which they started). And they can be programmed in a loop.
If you have an ability to use randomness - everything is simple. You can come up with many solutions. For example: with probability 0.5 each of them decide to that they will either do 3 steps right and wait. Or one step right and wait. If you will do this operation in a loop and they will select different options, then clearly they will meet (one is faster than the other, so he will reach a slower person). If they both select the same operation, than they will make a circle and both reach their starting positions. In this case roll the dice one more time. After N circles the probability that they will meet will be 1 - 0.5^n (which approaches 1 very fast)
Surprisingly, there is a way to do it! But first we have to define our terms and assumptions.
We have N=2 "teams" of K=2 "agents" apiece. Each "agent" is running the same program. They can't tell north from south, but they can tell clockwise from counterclockwise. Agents in the same place can talk to each other; agents in different places can't.
Your suggested partial answer was: "If each group splits in two and goes around and when they find another person keep on going with that person, they would all converge on the other side of the park..." This implies that our agents have some (magic, axiomatic) face-to-face decision protocol, such that if Alice and Bob are on the same team and wake up at the same point on the circle, they can (magically, axiomatically) decide amongst themselves that Alice will head clockwise and Bob will head counterclockwise (as opposed to Alice and Bob always heading in exactly the same direction because by definition they react exactly the same way to the situation they're identically in).
One way to implement this magic decision protocol is to give each agent a personal random number generator. Whenever 2 or more agents are gathered at a certain point, they all roll a million-sided die, and whichever one rolls highest is acknowledged as the leader. So in your partial solution, Alice and Bob could each roll: whoever rolls higher (the "leader") goes clockwise and sends the other agent (the "follower") counterclockwise.
Okay, having solved the "how do our agents make decisions" issue, let's solve the actual puzzle!
Suppose our teams are (Alice and Bob) and (Carl and Dave). Alice and Carl are the initially elected leaders.
Step 1: Each team rolls a million-sided die to generate a random number. The semantics of this number are "The team with the higher number is the Master Team," but of course neither team knows right now who's got the higher number. But Alice and Bob both know that their number is let's say 424202, and Carl and Dave both know that their number is 373287.
Step 2: Each team sends its leader around the circle clockwise, while the follower stays stationary. Each leader stops moving when he gets to where the other team's follower is waiting. So now at one point on the circle we have Alice and Dave, and at the other point we have Carl and Bob.
Step 3: Alice and Dave compare numbers and realize that Alice's team is the Master Team. Likewise, Bob and Carl compare numbers and realize that Bob's team is the Master Team.
Step 4: Alice being the leader of the Master Team, she takes Dave with her clockwise around the circle. Bob and Carl (being a follower and a leader of a non-master team respectively) just stay put. When Alice and Dave reach Bob and Carl, the problem is solved!
Notice that Step 1 requires that both teams roll a million-sided die in isolation; if during Step 3 everyone realizes that there was a tie, they'll just have to backtrack and try again. Therefore this solution is still probabilistic... but you can make its expected time arbitrarily small by just replacing everyone's million-sided dice with trillion-sided, quintillion-sided, bazillion-sided... dice.
The general strategy here is to impose a pecking order on all N×K agents, and then bounce them around the circle until everyone is aware of the pecking order; then the top pecker can just sweep around the circle and pick everyone up.
Imposing a pecking order can be done by using the agents' personal random number generators.
The protocol for K>2 agents per team is identical to the K=2 case: you just glom all the followers together in Step 1. Alice (the leader) goes clockwise while Bobneric (the followers) stay still; and so on.
The protocol for K=1 agents per team is... well, it's impossible, because no matter what you do, you can't deterministically ensure that anyone will ever encounter another agent. You need a way for the agents to ensure, without communicating at all, that they won't all just circle clockwise around the park forever.
One thing that would help with (but not technically solve) the K=1 case would be to consider the relative speeds of the agents. You might be familiar with Floyd's "Tortoise and Hare" algorithm for finding a loop in a linked list. Well, if the agents are allowed to move at non-identical speeds, then you could certainly do a "continuous, multi-hare" version of that algorithm:
Step 1: Each agent rolls a million-sided die to generate a random number S, and starts running clockwise around the park at speed S.
Step 2: Whenever one agent catches up to another, both agents glom together and start running clockwise at a new random speed.
Step 3: Eventually, assuming that nobody picked exactly the same random speeds, everyone will have met up.
This protocol requires that Alice and Carl not roll identical numbers on their million-sided dice even when they are across the park from each other. IMHO, this is a very different assumption from the other protocol's assuming that Alice and Bob could roll different numbers on their million-sided dice when they were in the same place. With K=1, we're never guaranteed that two agents will ever be in the same place.
Anyway, I hope this helps. The solution for N>2 teams is left as an exercise for the reader, but my intuition is that it'll be easy to reduce the N>2 case to the N=2 case.
Each group sends out a scout with the remaining group members remaining stationary. Each group remembers the name of their scout. The scouts circle around clockwise, and whenever he meets a group, they compare names of their scouts:
If scout's name is earlier alphabetically: group follows him.
If scout's name is later: he joins the group and gives up his initial group identity.
By the time the lowest named scout makes it back the his starting location, everyone who hasn't stopped at his initial location should be following him.
There are some solutions here that to me are unsatisfactory since they require the two teams to agree a strategy in advance and all follow the same deterministic or probabilistic rules. If you had the opportunity to agree in advance what rules you're all going to follow, then as flyx points out you could just have agreed a backup meeting point. Restrictions that prevent the advance choice of a particular place or a particular leader are standard in the context of some problems with computer networks but distinctly un-natural for four friends planning to meet up. Therefore I will frame a strategy from the POV of only one team, assuming that there has been no prior discussion of the scenario between the two teams.
Note that it is not possible to be robust in the face of any strategy from the other team. The other team can always force a stalemate simply by adopting some pattern of movement that ensures those two will never meet again.
One of you sets out walking around the park. The other stands still, let us say at position X. This ensures that: (a) you will meet each other periodically at X, let us say every T seconds; and (b) for each member of the other team, no matter how they move around the perimeter of the park they must encounter at least one of your team at least every T seconds.
Now you have communication among all members of both groups, and (given sufficient time and passing-on of messages from one person to another) the problem resolves to the same problem as if your mobile phones were working. Choosing a leader by random number is one way to solve it as others have suggested. Note that there are still two issues: the first is a two-generals problem with communication, and I suppose you might feel that a mobile phone conversation allows for the generation of common knowledge whereas these relayed notes do not. The second is the possibility that the other team refuses to co-operate and you cannot agree a meeting point no matter what.
Notwithstanding the above problems, the question supposes that if they had communication that the groups would be able to agree a meeting-point. You have communication: agree a meeting point!
As for how to agree a meeting point, I think it requires some appeal to reason or good intention on the part of the other team. If they are due to meet again, then they will be very reluctant to take any action that results in them breaking their commitment to their partner. Therefore suggest to them both that after their next meeting, when all commitments can be forgiven, they proceed together to X by the shortest route. Listen to their counter-proposal and try to find some common solution.
To help reach a solution, you could pre-agree with your team-mate some variations you'd be willing to make to your plan, provided that they remain within some restrictions that ensure you will meet your team-mate again. For example, if the stationary team-mate agrees that they could be persuaded to set out clockwise, and the moving team-mate sets out anti-clockwise and agrees that they can be persuaded to do something different but not to cross point X in a clockwise direction, then you're guaranteed to meet again and so you can accept certain suggestions from the other team.
Just as an example, if a team following this strategy meets a team (unwisely) following your strategy, then one of my team will agree to go along with the one of your team they meet, and the other will refuse (since it would require them to make the forbidden movement above). This means that when your team meet together, they'll have one of my team with them for a group of three. The loose member of my team is on a collision course with that group of three provided your team doesn't do anything perverse.
I think forming any group of three is a win, so each member should do anything they can to attend a meeting of the other team, subject to the constraints they agreed to guarantee they'll meet up with their own team member again. A group of 3, once formed, should follow whatever agreement is in place to meet the loose member (and if the team of two contained within that 3 refuses to do this then they're saboteurs, there is no good reason for them to refuse). Within these restrictions, any kind of symmetry-breaking will allow the team following these principles to persuade/follow the other team into a 3-way and then a 4-way meeting.
In general some symmetry-breaking is required, if only because both teams might be following my strategy and therefore both have a stationary member at different points.
Assume the park is a circle. (for the sake of clarity)
Group A
Person A.1
Person A.2
Group B
Person B.1
Person B.2
We (group A) are currently at the bottom of the circle (90 degrees). We agree to go towards 0 degrees in opposite directions. I'm person A.1 and I go clockwise. I send Person A.2. counterclockwise.
In any possible scenario (B splits, B doesn't split, B has the same scheme, B has some elaborate scheme), each group might have conflicting information. So unless Group A has a gun to force Group B into submission, the new groups might make conflicting choices upon meeting.
Say for instance, A.1. meets B.1, and A.2. meets B.2. What do we (A.1 and B.1) do if B has the same scheme? Since the new groups can't know what the other group decides (whether to go with A's scheme, or B's scheme), each group might make different decision.
And we'll end up where we started... (i.e. two people at 0 degrees, and two people at 90 degrees). Let's call this checkpoint "First Iteration".
We might account for this and say that we'll come up with a scheme for the "Second Iteration". But then the same thing happens again. And for the third iteration, fourth iteration, ad infinitum.
Each iteration has a 50% chance of not working out.
Which means that after x iterations, your chances of not meeting up at a common point are at most 1-(0.5^x)
N.B. I thought about a bunch of scenarios, such as Group A agreeing to come back to their initial point, and communicating with each other what Group B plans to do. But no cigar, turns out even with very clever schemes the conflicting information issue always arises.
An interesting problem indeed. I'd like to suggest my version of the solution:
0 Every group picks a leader.
1: Leader and followers go opposite directions
2: They meet other group leaders or followers
3: They keep going the same direction as before, 90 degrees magnitude
4: By this time, all groups have made a half-circle around the perimeter, and invariably have met leaders again, theirs, or others'.
5: All Leaders change the next step direction to that of the followers around,and order them to follow.
6: Units from all groups meet at one point.
Refer to the attached file for an in-depth explanation. You will need MS Office Powerpoint 2007 or newer to view it. In case you don't have one, use pptx. viewer (Powerpoint viewer) as a free alternative.
Animated Solution (.pptx)
EDIT: I made a typo in the first slide. It reads "Yellow and red are selected", while it must be "Blue and red" instead.
Each group will split in two parts, and each part will go around the circle in the opposite direction (clockwise and counterclockwise).
Before they start, they choose some kind of random number (in a range large enough so that there is no possibility for two groups to have the same number... or a Guid in computer science : globally unique identifier). So one unique number per group.
If people of the same group meet first (the two parts meet), they are alone, so probably the other groups (if any) gave up.
If two groups meet : they follow the rule that say the biggest number leads the way. So when they meet they continue in the direction that had people with the biggest number.
At the end, the direction of the biggest number will lead them all to one point.
If they have no computer to choose this number, each group could use the full names of the people of the group merged together.
Edit : sorry I just see that this is very close to Patrick Roberts' solution
Another edit : what if each group has its own deterministic strategy ?
In the solution above, all works well if all the groups have the same strategy. But in a real life problem this is not the case (as they cant communicate).
If one group has a deterministic strategy and the others have none, they can agree to follow the deterministic approach and all is ok.
But if two groups have deterministic approaches (simply for instance, the same as above, but one group uses the biggest number and the other group follows the lowest number).
Is there a solution to that ?
I was sick and so I missed my past 2 classes, I was wondering if someone could help me figure out how to solve this problem and I could sort of study it and try to understand it,I need pseudocode for this problem, I feel like I'm falling a little behind:
The Vernon Hills Mail-Order Company often sends multiple packages per order. For each customer order, output enough mailing labels to use on each of the boxes that will be mailed. The mailing labels contain the customer’s complete name and address, along with a box number in the form Box 9 of 9. For example, an order that requires three boxes produces three labels: Box 1 of 3, Box 2 of 3, and Box 3 of 3. Design an application that reads records that contain a customer’s title (for example, Mrs.), first name, last name, street address, city, state, zip code, and number of boxes. The application must read the records until eof is encountered and produce enough mailing labels for each order.
Write down each separate step that you list on a line of its own, and draw arrows between them, to indicate that a step should be followed by the next one.
That will process one "order". Since an order may consist of multiple boxes, look for where you can loop in this part. Draw a small arrow upwards to the right step where to restart for an individual box in an order.
At the end of this diagram you have processed a single "order", so now look for where the main loop should restart and on what condition.
With this done you have a flow chart; a purely visual aid, which you can translate into pseudocode (or, for that matter, directly into any programming language that has the right commands). So all that's left is to translate the graphic arrows into appropriate pseudo-code.
We have an auto-complete list that's populated when an you send an email to someone, which is all well and good until the list gets really big you need to type more and more of an address to get to the one you want, which goes against the purpose of auto-complete
I was thinking that some logic should be added so that the auto-complete results should be sorted by some function of most recently contacted or most often contacted rather than just alphabetical order.
What I want to know is if there's any known good algorithms for this kind of search, or if anyone has any suggestions.
I was thinking just a point system thing, with something like same day is 5 points, last three days is 4 points, last week is 3 points, last month is 2 points and last 6 months is 1 point. Then for most often, 25+ is 5 points, 15+ is 4, 10+ is 3, 5+ is 2, 2+ is 1. No real logic other than those numbers "feel" about right.
Other than just arbitrarily picked numbers does anyone have any input? Other numbers also welcome if you can give a reason why you think they're better than mine
Edit: This would be primarily in a business environment where recentness (yay for making up words) is often just as important as frequency. Also, past a certain point there really isn't much difference between say someone you talked to 80 times vs say 30 times.
Take a look at Self organizing lists.
A quick and dirty look:
Move to Front Heuristic:
A linked list, Such that whenever a node is selected, it is moved to the front of the list.
Frequency Heuristic:
A linked list, such that whenever a node is selected, its frequency count is incremented, and then the node is bubbled towards the front of the list, so that the most frequently accessed is at the head of the list.
It looks like the move to front implementation would best suit your needs.
EDIT: When an address is selected, add one to its frequency, and move to the front of the group of nodes with the same weight (or (weight div x) for courser groupings). I see aging as a real problem with your proposed implementation, in that it requires calculating a weight on each and every item. A self organizing list is a good way to go, but the algorithm needs a bit of tweaking to do what you want.
Further Edit:
Aging refers to the fact that weights decrease over time, which means you need to know each and every time an address was used. Which means, that you have to have the entire email history available to you when you construct your list.
The issue is that we want to perform calculations (other than search) on a node only when it is actually accessed -- This gives us our statistical good performance.
This kind of thing seems similar to what is done by firefox when hinting what is the site you are typing for.
Unfortunately I don't know exactly how firefox does it, point system seems good as well, maybe you'll need to balance your points :)
I'd go for something similar to:
NoM = Number of Mail
(NoM sent to X today) + 1/2 * (NoM sent to X during the last week)/7 + 1/3 * (NoM sent to X during the last month)/30
Contacts you did not write during the last month (it could be changed) will have 0 points. You could start sorting them for NoM sent in total (since it is on the contact list :). These will be showed after contacts with points > 0
It's just an idea, anyway it is to give different importance to the most and just mailed contacts.
If you want to get crazy, mark the most 'active' emails in one of several ways:
Last access
Frequency of use
Contacts with pending sales
Direct bosses
Etc
Then, present the active emails at the top of the list. Pay attention to which "group" your user uses most. Switch to that sorting strategy exclusively after enough data is collected.
It's a lot of work but kind of fun...
Maybe count the number of emails sent to each address. Then:
ORDER BY EmailCount DESC, LastName, FirstName
That way, your most-often-used addresses come first, even if they haven't been used in a few days.
I like the idea of a point-based system, with points for recent use, frequency of use, and potentially other factors (prefer contacts in the local domain?).
I've worked on a few systems like this, and neither "most recently used" nor "most commonly used" work very well. The "most recent" can be a real pain if you accidentally mis-type something once. Alternatively, "most used" doesn't evolve much over time, if you had a lot of contact with somebody last year, but now your job has changed, for example.
Once you have the set of measurements you want to use, you could create an interactive apoplication to test out different weights, and see which ones give you the best results for some sample data.
This paper describes a single-parameter family of cache eviction policies that includes least recently used and least frequently used policies as special cases.
The parameter, lambda, ranges from 0 to 1. When lambda is 0 it performs exactly like an LFU cache, when lambda is 1 it performs exactly like an LRU cache. In between 0 and 1 it combines both recency and frequency information in a natural way.
In spite of an answer having been chosen, I want to submit my approach for consideration, and feedback.
I would account for frequency by incrementing a counter each use, but by some larger-than-one value, like 10 (To add precision to the second point).
I would account for recency by multiplying all counters at regular intervals (say, 24 hours) by some diminisher (say, 0.9).
Each use:
UPDATE `addresslist` SET `favor` = `favor` + 10 WHERE `address` = 'foo#bar.com'
Each interval:
UPDATE `addresslist` SET `favor` = FLOOR(`favor` * 0.9)
In this way I collapse both frequency and recency to one field, avoid the need for keeping a detailed history to derive {last day, last week, last month} and keep the math (mostly) integer.
The increment and diminisher would have to be adjusted to preference, of course.