I'm having an error in my Laravel 8 app which uses yajra/laravel-datatables-oracle with server-side rendering, and I can't figure it out so far. What makes it frustating is the fact that this have been working just fine for the last couple of months, and suddenly I got a report about this error just this morning.
So I have a page which shows a table. The table itself is pretty large, with more than 25 columns shown, thousands of total records, and with some nested relationships as well. For this, I use the yajra/laravel-datatables-oracle package with ajax/server-side rendering as I mentioned above. The page has pagination option enabled, and is set to show 50 records by default. The page is functioning correctly when opened in its default condition.
However, an error will happen when user changes the table's page-length to "All". After checking the the route that handles the table's ajax rendering, turns out the error happened because the returned json is not a valid json. So instead of getting a valid json like these:
"data": [
{"id": 1, "name": "Foo"},
// ....
{"id": 999, "name": "Bar"}
]
I get:
"data": [
{"id": 1, "name": "Foo"},
// ....
{"id": 500,
]
// or...
"data": [
{"id": 1, "name": "Foo"},
// ....
{"id": 500, "name": "Ja
]
The point is that the returned response is incomplete, which makes it invalid as a json, which makes the datatables throws a json bad parsing error.
These are the things for you to know:
Remember, this thing have been working fine for the last couple of months.
Just like the example above, the json response (which is incomplete) is always changing everytime I refresh the url. Meaning that it's likely not related to "abnormal" record.
The error occurs only if the selected page-lenght pass a certain amount of number:
It's working when just being selected (up to) 180 items per page (each item has a total of >100 fields).
More than 180 and the error will occur. However, the incompleteness could be anywhere, not in the 181'th item. It could be in the 70th-ish, 90th-ish, etc. In fact, It never returns more than 100 records when the error occur.
I had thought that this might be caused by PHP's memory exhaustion. I tried to set the memory_limit of the PHP to -1 or 999999M, but nothing changes.
This is actually not just happening in one page, but many.
However, there are another pages, that's working totally fine without the error above. The size of the data that it has is roughly the same.
I really have no idea what's the cause of this. Is this because of the server's PHP config, the database (mysql), the ajax request, or what?
Please help. You're much appreciated.
Additional Info
This is one of the method's controller where the error appears.
use App\Models\Instansi;
use App\Models\InstrukturKelasPeriodeTahapan;
use App\Models\Tahapan;
use App\Models\Tahun;
use Illuminate\Http\Request;
use Yajra\DataTables\Facades\DataTables;
public function datatable(Request $request) {
// validate request etc...
$jadwals = InstrukturKelasPeriodeTahapan::query()
->withInstruktur()->withKelasPeriodeTahapan()
->withTarifInTahunAndPeriodeOf($tahun, $periode)
->withPphInTahunAndPeriodeOf($tahun, $periode)
->withCount([
'pamongDalamKelasPeriodeTahapan as jumlah_pamong_yg_mengajar'
])
->whereHasTahunOf($tahun)
->whereHasPeriodeOf($periode)
->whereHasTahapanOf(Tahapan::PPP);
if ($request->instansi) {
if ($request->instansi == Instansi::NON_FOO) {
$jadwals->whereHasInstansiOfNonFoo();
} else {
$jadwals->whereHasInstansiOf($request->instansi);
}
}
return DataTables::of($jadwals)
->addIndexColumn()
->make(true);
}
This should get a total of 1134 records and used to be working just fine. Now, when the error occurs, I get less than 100 records, and the last returned record is incomplete.
These different return syntaxes still giving me incomplete result.
return response()->json(
DataTables::of($jadwals)
->addIndexColumn()
->make(true)
);
return DataTables::of($jadwals)
->addIndexColumn()
->toJson();
return $jadwals->get();
return response()->json($jadwals->get());
However, these sintaxes is returning the complete 1134 records. But I can't use it as it's not using datatable's response format.
return dd($jadwals->get());
return $jadwals->get()->toJson();
And this is a data example of how an item/record should look like when returned completely, with 120 lines in total:
{
"id": 9999,
"kelas_periode_tahapan_id": 999,
"instruktur_id": 999,
"jumlah_pamong_yg_mengajar": 9,
"instruktur": {
"id": 9999,
"id_ref": 9999,
"nama": "Dr. Xxxxx",
"email": "xxxxxxxx#gmail.com",
"tgl_lahir": "95-09-9977",
"nip": "999999999999999999",
"nik": "99999999999999999",
"npwp": "99999999999999999",
"golongan_id": 9,
"jab_fungsional_id": 9,
"instansi_id": 9,
"no_rek": "99999999999",
"nama_bank": "BANK XXXXXXXX",
"bank_cabang": "XXX",
"nama_rek": "Xxxxxxxx Xxxxx Xxxxxxxxx",
"is_active": 9,
"golongan": {
"id": 9,
"description": "XXXXX",
"periodes": [
{
"id": 9,
"tahun_id": 9,
"periode": 9,
"pivot": {
"golongan_id": 9,
"periode_id": 9,
"pph_npwp": 9,
"pph_non_npwp": 9
}
}
]
},
"jab_fungsional": {
"id": 9,
"description": "Xxxxxxxx",
"periodes": [
{
"id": 9,
"tahun_id": 9,
"periode": 9,
"pivot": {
"jab_fungsional_id": 9,
"periode_id": 9,
"tarif": 99999999
}
}
]
},
"instansi": {
"id": 9,
"description": "Xxxxxxxxxxx Xxxxxxxxxxxxx Xxxxxxxxxxxx"
}
},
"kelas_periode_tahapan": {
"id": 999,
"kelas_id": 99,
"periode_tahapan_id": 99,
"is_verified": 9,
"is_locked": 9,
"kelas": {
"id": 99,
"periode_prodi_id": 9,
"description": "Xxxxxxxxxxx Xxxxxxxxxxxxx Xxxxxxxxxxxx",
"nama_grup_wa": "Xxxxxxxx Xxxxxxxxxxx",
"link_grup_wa": "https://xxxxxxxxxxxxxxxxxxxxxxxxx",
"periode_prodi": {
"id": 9,
"prodi_id": 9,
"periode_id": 9,
"prodi": {
"id": 9,
"nama": "Xxxxxxxxxxxxx Xxxxxxxxxxxxx",
"kode": "999",
"is_active": 9
}
}
},
"periode_tahapan": {
"id": 99,
"periode_id": 9,
"tahapan_id": 9,
"ket_hari_ke": 99,
"jadwal": "09-08-9099",
"materi": "Xxxxxxxxxxx Xxxxxxxx Xxxxxxxxxxxxxxxxxxxxx",
"jp_full": 9,
"jp_pamong": 9,
"tahapan": {
"id": 9,
"description": "Xxxxxxxxxxx Xxxxxxxxxxxxx Xxxxxxxxxxxx",
"akronim": "XXX",
"satuan": "XXX"
},
"periode": {
"id": 9,
"tahun_id": 9,
"periode": 9,
"tahun": {
"id": 9,
"tahun": 9099
},
"persentase_honor": {
"id": 9,
"periode_id": 9,
"full_dosen": 900,
"full_gpamong": 900,
"persentase_dosen": 99,
"persentase_gpamong": 99
}
}
}
},
"DT_RowIndex": 1
}
Here is my code.
$orders = Order::with('customer')->withCount('orderItems')->paginate(50)->toArray();
This code return me the blank page. No data return. So here's what I did wrong?
When you are using Laravel pagination, it simply gives you an array of Data.
You don't need to convert to toArray().Use the following code :
$orders = Order::with('customer')->withCount('orderItems')->paginate(15);
The response of the pagination will be the following :
Content-Type: application/vnd.api+json
{
"meta": {
"page": {
"current-page": 2,
"per-page": 15,
"from": 16,
"to": 30,
"total": 50,
"last-page": 4
}
},
"links": {
"first": "http://localhost/api/v1/posts?page[number]=1&page[size]=15",
"prev": "http://localhost/api/v1/posts?page[number]=1&page[size]=15",
"next": "http://localhost/api/v1/posts?page[number]=3&page[size]=15",
"last": "http://localhost/api/v1/posts?page[number]=4&page[size]=15"
},
"data": [...]
}
I'm using Laravel 8.20.1 and my API route returns a paginated JSON response.
Is there a better way than str_replace() to remove the arrows from the labels?
routes/api.php:
Route::middleware('auth:sanctum')->get('/items', function (Request $request) {
return new ItemCollection(Item::paginate(5));
});
response.data.meta
{
"current_page": 1,
"from": 1,
"last_page": 3,
"links": [
{
"url": null,
"label": "« Previous",
"active": false
},
....
{
"url": "http://localhost/api/items?page=2",
"label": "Next »",
"active": false
}
],
"path": "http://localhost/api/items",
"per_page": 5,
"to": 5,
"total": 15
}
Those labels for Next and Previous are in the translation files, resources/lang/en/pagination.php. If you look at Illuminate\Pagination\LengthAwarePaginator::linkCollection you can see it building the links part of the response.
I am building an API Gateway and I am having a small problem returning the pagination urls from a microservice to the API Gateway.
This is the current structure of my API Gateway:
When I call the microservice, I can easily pass the paging parameters using the request data:
HTTP::get('http://api.billing.microservice.test/v2/invoices', $request->all());
However, when I make a request for a microservice, it returns the requested data, but with the URL of the microservice:
{
"data": [
# data returned from billing microservice with the billing API URL
],
"links": {
"first": "http://api.billing.microservice.test/v2/invoices?page=1",
"last": "http://api.billing.microservice.test/v2/invoices?page=10",
"prev": null,
"next": "http://api.billing.microservice.test/v2/invoices?page=2"
},
"meta": {
"current_page": 1,
"from": 1,
"last_page": 10,
"path": "http://api.billing.microservice.test/v2/invoices",
"per_page": 30,
"to": 30,
"total": 300
}
}
However I need the return to have the main API URL:
{
"data": [
# data returned from billing microservice with the main API Address
],
"links": {
"first": "http://api.main.test/v2/invoices?page=1",
"last": "http://api.main.microservice.test/v2/invoices?page=10",
"prev": null,
"next": "http://api.main.test/v2/invoices?page=2"
},
"meta": {
"current_page": 1,
"from": 1,
"last_page": 10,
"path": "http://api.main.test/v2/invoices",
"per_page": 30,
"to": 30,
"total": 300
}
}
Has anyone had to do something similar? What is the best way to achieve the desired results? Do a replace using some kind of regex? Is there anything I can do inside the microservice?
I was able to solve my problem with a simple method that already exists:
Model::paginate()->setPath('http://api.main.microservice.test/v2/invoices');
The result is:
{
"data": [
# data
],
"links": {
"first": "http://api.main.microservice.test/v2/invoices?page=1",
"last": "http://api.main.microservice.test/v2/invoices?page=1",
"prev": null,
"next": null
},
"meta": {
"current_page": 1,
"from": 1,
"last_page": 1,
"path": "http://api.main.microservice.test/v2/invoices",
"per_page": 30,
"to": 3,
"total": 3
}
}
I'm developing an API, and for index routes, I return paginated but now I should using eloquent resources, but pagination JSON structure of these two are different and front-end code doesn't work anymore.
my codes are:
// AdminUserController.php
public function index()
{
return User::paginate();
}
// ClientUserController.php
public function index()
{
return new UserCollection(User::paginate());
}
the first JSON structure like :
{
"total": 50,
"per_page": 15,
"current_page": 1,
"last_page": 4,
"next_page_url": "http://my.app/api/admin/users?page=2",
"prev_page_url": null,
"from": 1,
"to": 15,
"data": [
{
// Result Object
},
{
// Result Object
}
]
}
But the second is like:
{
"data": [
{
// Result Object
},
{
// Result Object
}
],
"links":{
"first": "http://my.app/api/app/users?page=1",
"last": "http://my.app/api/app/users?page=2",
"prev": null,
"next": null
},
"meta":{
"current_page": 1,
"from": 1,
"last_page": 2,
"path": "http://my.app/api/admin/users",
"per_page": 15,
"to": 15,
"total": 25
}
}
why laravel paginated responses have different JSON structure?
What does your UserCollection class look like?
At first glance, it looks like this is because the first method AdminUserController#index is converting the response to JSON, while the second method ClientUserController #index is returning a collection.