Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat 1256 runner to not use meetup api to display events #1335

Merged
1 change: 0 additions & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@
"smarty/smarty": "2.6.*",
"sabre/vobject": "^4.1",
"erusev/parsedown": "^1.6",
"dms/meetup-api-client": "^2.3",
"google/apiclient": "^2.0",
"robmorgan/phinx": "^0.9.2",
"presta/sitemap-bundle": "^1.5",
Expand Down
64 changes: 3 additions & 61 deletions composer.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

89 changes: 39 additions & 50 deletions sources/AppBundle/Indexation/Meetups/Runner.php
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,11 @@
namespace AppBundle\Indexation\Meetups;

use AlgoliaSearch\Client;
use AppBundle\Offices\OfficesCollection;
use DMS\Service\Meetup\MeetupOAuthClient;
use AppBundle\Event\Model\Meetup;
use AppBundle\Event\Model\Repository\MeetupRepository;
use CCMBenchmark\Ting\Repository\CollectionInterface;
use Symfony\Component\Process\Exception\ProcessFailedException;
use Symfony\Component\Process\Process;

class Runner
{
Expand All @@ -14,26 +17,14 @@ class Runner
protected $algoliaClient;

/**
* @var MeetupOAuthClient
* @var MeetupRepository
*/
protected $meetupClient;
protected $meetupRepository;

/**
* @var OfficesCollection
*/
protected $officiesCollection;

/**
* @var Transformer
*/
protected $transformer;

public function __construct(Client $algoliaClient, MeetupOAuthClient $meetupClient)
public function __construct(Client $algoliaClient, MeetupRepository $meetupRepository)
{
$this->algoliaClient = $algoliaClient;
$this->meetupClient = $meetupClient;
$this->officiesCollection = new OfficesCollection();
$this->transformer = new Transformer($this->officiesCollection);
$this->meetupRepository = $meetupRepository;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pour moi il faut conserver l'appel au transformer. Le format de ce qu'on envoie à Algolia doit toujours être celui-ci :

public function transform(array $meetup)
(mais la source doit être la base de données et plus l'api meetup)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok je modifie ça à l'occasion

Copy link
Contributor Author

@vinceAmstoutz vinceAmstoutz Nov 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

C'est fait j'ai restoré le transformer par contre j'ai des doutes sur le faite qu'il fasse lui passer OfficesCollections à l'instanciation. Le contenu de la méthode transform me parait également peut adapté puis qu'on a un tableau d'objet Meetups désormais à traiter. Comment puis-je tester tout ca @agallou ? En modifiant tests/units/AppBundle/Indexation/Meetups/Transformer.php ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Le plus simple que je vois est de se créer un compte de test sur Algolia.
De modifier les paramètres lié à algolia dans le parameters.yml :

algolia_app_id: DVB92YWTPE

Et de relancer indexing:meetups.
Et ensuite aller voir cette page en local : https://afup.org/meetups/
Pour moi le officesCollection peux rester, il permet d'en récupérer les libellés, mais effectivement il faut adapter le transformer pour non plus traiter un tableau qui viens de l'API meetup, mais de la base de données.
(au besoin si tu veux on peux s'appeler si ça peux être plus clair)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Je re-regarde ca dès que possible et je te redis

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Voilà c'est fait @agallou 🙂
image
image

}

/**
Expand All @@ -43,44 +34,19 @@ public function run()
{
$index = $this->initIndex();

$command = $this->meetupClient->getCommand(
'GetEvents',
[
'group_id' => implode(',', $this->getGroupIds()),
'status' => 'upcoming,past',
'order' => 'time',
'desc' => 'true',
]
);

$command->prepare();

$meetups = [];

foreach ($command->execute() as $meetup) {
if (null === ($transformedMeetup = $this->transformer->transform($meetup))) {
continue;
}
$meetups[] = $transformedMeetup;
$process = new Process(['php', 'bin/console', 'scraping:meetup:event']);
$process->run();

if (!$process->isSuccessful()) {
throw new ProcessFailedException($process);
}

$meetups = $this->getMeetupsFromDatabase();

$index->clearIndex();
$index->addObjects($meetups, 'meetup_id');
}

protected function getGroupIds()
{
$groupIds = [];
foreach ($this->officiesCollection->getAll() as $office) {
if (!isset($office['meetup_id'])) {
continue;
}
$groupIds[] = $office['meetup_id'];
}

return $groupIds;
}

/**
* @return \AlgoliaSearch\Index
*/
Expand All @@ -106,4 +72,27 @@ protected function initIndex()

return $index;
}

/**
* @return array
*/
private function getMeetupsFromDatabase()
{
$meetupsCollection = $this->meetupRepository->getAll();

return $this->fromCollectionInterfaceToArray($meetupsCollection);
}

/**
* @param CollectionInterface $meetupsCollection
* @return array<Meetup>
*/
public function fromCollectionInterfaceToArray($meetupsCollection)
{
$meetupsArray = [];
foreach ($meetupsCollection as $meetup) {
$meetupsArray[] = $meetup;
}
return $meetupsArray;
}
}
Loading