Doctrine Query: Select from associations - filter

Question

i am trying to create the a query for the following situation, but first the used entities:
User:
/**
* Friendships to me (the user)
*
* #OneToMany(targetEntity="Friendship", mappedBy="to", fetch="EAGER")
*/
protected $friendshipsTo;
/**
* Friendships from me (the user)
*
* #OneToMany(targetEntity="Friendship", mappedBy="from", fetch="EAGER")
*/
protected $friendshipsFrom;
Friendship:
/**
* The user who started the friend request
*
* #var \Entity\User
* #ManyToOne(targetEntity="User", inversedBy="friendshipsFrom")
*/
protected $from = null;
/**
* The user who got the friend request
*
* #var \Entity\User
* #ManyToOne(targetEntity="User", inversedBy="friendshipsTo")
*/
protected $to = null;
I hope this pretty much explains it. If I add a friend, a new friendship entity will be created with me as "from" and the other user as "to".
So now I want to select all friends of a user. I already figured that out, here's my solution:
$query = $qb->select('u, ff, ft')
->from('KontakteApp\Api\Entity\User', 'u')
->leftJoin('u.friendshipsTo', 'ft')
->leftJoin('u.friendshipsFrom', 'ff')
->where('ft.from = :user')
->orWhere('ff.to = :user')
->andWhere('u.id != :user')
->getQuery();
So what's the problem now? I realized that in this case if I find a friend (here id = 3) of the user with the id 2 with
{
"id": 3,
"friendshipFrom": [
{
"id": 42,
"from": 3,
"to": 2,
"status": "open"
},
**HERE ARE NO MORE FRIENDSHIPS**
],
"friendshipTo": [
This is array is full of friendships of this user
]
}
there are no more friendships in friendshipsFrom except the one which had the user show up in my select. This happens because of select('u, ff, ft').
Now I would like the same happen to friendshipsTo, so in this case friendshipsTo of the friend should be empty.
To sum up: How do I filter associated entities?