Images don't load if I have a lot of items in the database.

[plyaskin]

Terms

• Before you openning this issue, i have checked if the issue has already been reported.

Bagisto Version(s) affected

2.1.2

Issue Description

I have a separate DB and Linux server in a local network:
DB is [4 x 3 GHz CPU • 8 GB RAM • 80 GB NVMe] and
Linux server is [1 x 3 GHz CPU • 2 GB RAM • 15 GB NVMe]
I filled items to the DB with DataFaker, there are about 70k items and any images in the Shop don't load.

Preconditions

Filling items to the DB with DataFaker

Steps to reproduce

To fill 50k+ items to the DB with DataFaker

Expected Result

It works stability

Actual Result

Images don't load

Screenshots

Additional context

How many items is a maximum for Bagisto?
70k items are too low for the eCommerce and well-designed MySQL DB.
Redis and Octane don't help with performance for this issue.
Can I use MongoDB or any other way for high performance?

[jitendra-webkul]

To set up Elasticsearch, follow these steps:

1. Configure Elasticsearch:
   Use the configuration file from the following link:
   https://github.com/bagisto/bagisto/blob/master/config/elasticsearch.php

2. Enable Elasticsearch in the Admin Panel:
   Go to "Configure >> Products >> Storefront >> Search Mode" and select "Search Mode => Elastic Search".

3. Index Products in Elasticsearch:
   Run the following command to index existing products:

   php artisan indexer:index --type=elastic --mode=full

   After the initial setup, indexing will automatically occur during product creation, update, and deletion.

[jitendra-webkul]

The image load issue is not related to the Faker package. It might be caused by another problem. Try creating symlinks to the storage directory using the following command:

php artisan storage:link

[plyaskin]

To set up Elasticsearch, follow these steps:

1. Configure Elasticsearch:
   Use the configuration file from the following link:
   https://github.com/bagisto/bagisto/blob/master/config/elasticsearch.php

2. Enable Elasticsearch in the Admin Panel:
   Go to "Configure >> Products >> Storefront >> Search Mode" and select "Search Mode => Elastic Search".

3. Index Products in Elasticsearch:
   Run the following command to index existing products:

   php artisan indexer:index --type=elastic --mode=full

   After the initial setup, indexing will automatically occur during product creation, update, and deletion.

Could you explain how this search engine can help with the database and images?

[jitendra-webkul]

As mentioned in my previous reply, if you have a large catalog, you should use Elasticsearch. With Elasticsearch, products are filtered using Elasticsearch rather than the database. The schema for storing product information is well normalized, and pulling information from multiple tables can cause performance issues. This is why we provided support for Elasticsearch. In the future, we will remove database support for products on the frontend, allowing only Elasticsearch or OpenSearch.

Regarding the image issue, it is not related to the items. It's possible that files have been removed from the storage folder, or there may be another issue. Please check the paths of the broken images to see if the files exist in the storage folder.

[plyaskin]

As mentioned in my previous reply, if you have a large catalog, you should use Elasticsearch. With Elasticsearch, products are filtered using Elasticsearch rather than the database. The schema for storing product information is well normalized, and pulling information from multiple tables can cause performance issues. This is why we provided support for Elasticsearch. In the future, we will remove database support for products on the frontend, allowing only Elasticsearch or OpenSearch.

Regarding the image issue, it is not related to the items. It's possible that files have been removed from the storage folder, or there may be another issue. Please check the paths of the broken images to see if the files exist in the storage folder.

The files are in the storage because I didn't touch them and they appear when the items are less in DB.

[jitendra-webkul]

Can you share you instance URL so that I can check

[plyaskin]

Unfortunately, I have already removed it.

[jitendra-webkul]

No problem, let me try to reproduce the issue on my end by creating 50,000 products.

[plyaskin]

No problem, let me try to reproduce the issue on my end by creating 50,000 products.

Please check about 1 million products also.

[jitendra-webkul]

We have already tested with more than 1 million and elastic search.

[jitendra-webkul]

The lazy image loading was causing the issues. I've addressed this in issue #9823. Please pull the latest changes from the master branch and verify.

[plyaskin]

We have already tested with more than 1 million and elastic search.

Could you let me know how to use the Elastic?
This Article isn't applicable.

[jitendra-webkul]

You can follow the documentation provided here:

https://devdocs.bagisto.com/2.x/advanced/indexing-products-to-elasticsearch.html#indexing

The images issue has already been fixed, so I am closing this issue.