I am trying to think of a way to reduce my code base for 2 similar payment modules I am developing. The two models share some logic, which could be abstracted with the use of admin config options.
For example, both modules are used to calculate instalments. Module 1 specification dictates that maximum number of instalments is 6 with 4% interest rate, but Module 2 specification allows for up to 12 instalments with 6% interest. It would be of additional benefit if I could dynamically add N instances of the module.
Now, my development environment consists of multiple repositories (each module has its own repository) and everything is connected with composer. I would like this module to live in one directory and an easy way to "duplicate" (by that I mean: display new payment options in the checkout, with its own setting) the module.
I hope the question is clear enough. If not, please tell me what information would you need to understand it better.
Related
I am trying to implement a gem5 version of HybCache as described in HYBCACHE: Hybrid Side-Channel-Resilient Caches for Trusted Execution Environments (which can be found at https://www.usenix.org/system/files/sec20spring_dessouky_prepub.pdf).
A brief summary of HybCache is that a subset of all the cache is reserved for use by secure processes and are isolated. This is achieved by using a limited subset of cache ways when the process is in 'isolated' mode. Non-isolated processes uses the cache operations normally, having access to the entire cache and using the replacement policy and associativity given in the configuration. The isolated subset of cache ways uses random replacement policy and is fully associative. Here is a picture demonstrating the idea.
The ways 6 and 7 are grey and represent the isolated cache ways.
So, I need to manipulate the placement of data into these ways. My question is, since I have found no mention of cache ways in the gem5 code, does that mean that the cache ways only exist logically? That is, do I have to manually calculate the location of each cache way? If cache ways are used in gem5, then were are they used? What is the file name?
Any help would be greatly appreciated.
This answer is only valid for the Classic cache model (src/mem/cache/).
In gem5 the number of cache ways is determined automatically from the cache size and the associativity. Check the files in src/mem/cache/tags/indexing_policies/ for the relevant code (specifically, the constructor of base.cc).
There are two ways you could tackle this implementation:
1 - Create a new class that inherits from BaseTags (e.g., HybCacheTags). This class will contain the decision of whether it should work in secure mode or not, and how to do so (i.e., when to call which indexing and replacement policy). Depending on whatever else is proposed in the paper, you may also need to derive from Cache to create a HybCache.
The new tags need one indexing policy per operation mode. One is the conventional (SetAssociative), and the other is derived from SetAssociative, where the parameter assoc makes the numSets become 1 (to make it fully associative). The derived one will also have to override at least one function, getPossibleEntries(), to only allow selecting the ways that you want. You can check skewed_assoc.cc for an example of a more complex location selection.
The new tags need one replacement policy per operation mode. You will likely just use the ones in the replacement_policies folder.
2 - You could create a HybCache based on the Cache class that has two tags, one conventional (i.e., BaseSetAssoc), and the other based on the FALRU class (rewritten to work as a, e.g., FARandom).
I believe the first option is easier and less hardcoded. FALRU has not been split into an indexing policy and replacement policy, so if you need to change one of these, you will have to reimplement it.
While implementing you may encounter coherence faults. If it happens it is much likely a problem in the indexing logic, and I wouldn't look into trying to find issues in the coherence model.
I've just started a new job and we have several installs of magneto all of different versions!
Now really it seems to me that we need to firstly upgrade them all and then get them all under one installation of magneto and using one database.
What is the best way (in general terms) of doing this.
Is it even possible or is my best bet to make the sites again under one installation and import the products into it.
There is some talk by a fellow developer that having them under different installs helps with performance. Is this true?
Once we have them all under one install things like stock control and orders as well as putting products on multiple site should also be very straightforward - correct?
We are talking quite a few stores say around 15ish and quite a few products around I would say 4000 maybe more.
My first suggestion is to consider the reasons, why you need all Magento instances to be moved under one installation. The reasons are not clear from your question. So the best developer's advice is "Does it work? Then don't touch it" :)
If there are no specific reasons, then you'd better leave it as is. All reorganization processes (upgrading, infrastructure configuration, access setup, etc.) for a software system are hard, costly, consume time, error-prone, usually have no much value from business point of view and are a little boring. This is not a Magento-specific thing, it is just general characteristic for any software.
Also note, that it is a holiday season. So it is better not to do anything with e-commerce stores until the middle of January.
If you see value in a reorganization of your Magento stores, then the best way to do it is to go gradually - step by step, store by store:
Take your most complex store. Prepare everything you need for the further steps - i.e. get ready the tools, write automatic scripts, go through the process with its copy at some testing server. Write set of functional tests
to cover it with at least smoke-tests. You'll have to repeat
such light checks many times to be sure, that the store appears to be
working. The automatic tests will save much time. Thus all these preparations will decrease your downtime.
Close public access to the store.
Upgrade store to the Magento version, you need. Move it to the new infrastructure.
Verify all the user scenarios manually and with automatic tests. Fix the issues, if any.
Open public access to the store. Monitor logs, load reports at the server machines. Fix issues, if any.
Take next store (let's call it NextStore). Make its copy at a sandbox server.
Make copy of your already converted store (let's call it ConvertedStore) at a sandbox server.
Export all the data from copy of NextStore and import it to the copy of ConvertedStore. You can use Magento Dataflow or Import/Export modules to do that. Not all data can be
imported/exported with those modules - just Catalog, Orders, Customers. You will need
to develop custom scripts to import/export other entities, if you need them.
Verify result manually and with automatic tests and manually. Write automatic scripts, that fix found issues. You will need those scripts later during the real converting process.
Close NextStore.
Move it to the new infrastructure, by engaging the already prepared procedures and scripts. You will need to consider, whether to close ConvertedStore during the converting process. It depends on your feeling, whether it is ok to have it opened or not. For safety reasons it is better to close it.
Verify, that everything works fine. Monitor logs, reports.
Fix issues, if any.
Proceed with the rest of your stores.
That is my (totally personal) view on the procedures.
There is some talk by a fellow developer that having them under
different installs helps with performance. Is this true?
Yes, your friend is right. Separating Magento (actually, anything in this world) into smaller instances makes it lighter to be handled. The performance difference is very small (for your instance of 4000 products), but it is inevitable. Consider, that after combining the instances (suppose, there are ten of them with 400 products each) you'll be handling data for 10x more customers, reports, products, stores, etc. Therefore any search will have to go through ten times more products, in order to return data. Of course, it doesn't matter, if the search takes 0.00001 second, because 0.0001s for combined instance is ok as well. But some things, like sorting or matching sets, grow non-linearly. But, as said before, for 4000 products you won't see big difference.
Once we have them all under one install things like stock control and
orders as well as putting products on multiple site should also be
very straightforward - correct?
You're right - after combining the stores together, handling orders, stock, customers will be much more simpler and straightforward process.
Good luck! :)
The most important thing to consider is what problem you're solving by having all these sites on one Magento "instance". What's more important to your business/team: having these sites share product and inventory or having the flexibility of independently modifying these sites? Any downtime or impacts to availability may affect all sites.
Further questions/areas of investigation:
How much does the product hierarchy (categories and attributes) differ?
Is pricing the same across each site or different?
Are any of these sites multi-regional and how is pricing handled for each region?
It's certainly possible to run multiple sites on one Magento instances, even if there are some rough edges within the platform.
Since there's no way to export all entities in Magento, there's no functionality to merge stores. You'd have to write custom code - it would have to take all the records from the old store, assign them new IDs while preserving referential integrity & insert them into the new store (this is what the "product import" does, but they don't have it for categories, orders, customers, etc.).
The amount of code you'd be writing to do that would take almost longer than just starting over in my opinion. You'd basically be writing the missing functionality for Magento. If it were easy they would have it done it already.
However splitting two stores apart is very easy, since you don't have to worry about reassigning unique identifiers in the DB.
in our website www.theprinterdepo.com we are going to implement google checkout. However I am not sure in what shipping methods or strategy to use.
In this page:
https://developers.google.com/checkout/developer/Google_Checkout_XML_API_Carrier_Calculated_Shipping#Process
Google says that they calculate based on the total weight of the items, but the thing is if one person buys one printer thats fine, but if he orders 3 printers of 50lbs, the shipping cost is invalid calculating it with 150lbs. It has to be calculated as 3 packages of 50lbs.
How would you do it in this scenario??
I have only had minimal investigation to this, but I don't think this can be handled by default installation. I know that you would need a shipping extension that can support the Google API shipping-packages, but real issue is that not even the Google API can support more than one package, either by API limitation or restriction by choice.
The <shipping-packages> tag encapsulates information about
all of the packages that will be shipped to the buyer.
At this time, merchants may only specify one package per order
I would love to see this come to full use as it would be a great addition to be able to say that anything with a weight over x requires additional packaging but currently I don't think it is possible. While this can be accomplished by separating the order into three orders, but that will over complicate the user experience and possible cause loss of sales.
Source:
https://developers.google.com/checkout/developer/Google_Checkout_XML_API_Carrier_Calculated_Shipping#tag_shipping-packages
The "limitation" mentioned above is only if you will rely on Google to calculate shipping for you using what they call carrier-calculated-shipping.
You do have other options to calculate shipping:
you can pre-calculate using whatever formula (or shipping service/plugin) you have based on the cart contents (you would know this prior to handing off the cart to Google for Checkout), which is essentially sending a flat rate shipping cost to Google, or perhaps;
use the merchant-calculations-api option so you can account for the destination/delivery address (not just cart contents). This option is more complex (you need to handle callbacks from Google), but it does give you critical information to work with when calculating shipping.
hth....
Suppose I have to create functionalities A, B and C through custom coding in Drupal using hooks.
Either I can club three of them in custom1.module or I can create three separate modules for them, say custom1.module, custom2.module and custom3.module.
Benefits of creating three modules:
Clean code
Easily searchable
Mutually independent
Easy to commit in multi-developer projects
Cons:
Every module entry gets stored in the database and requires a query.
To what extent does it mar the performance of the site?
Is it better to create a single large custom module file for the sake of reducing database queries or break it into different smaller ones?
This issue might be negligible for small scale websites, let the case be for large scale performance oriented sites.
I would code it based on how often do I need function A , B and C
Actual example:
I made a module which had two needs
1) Send periodic emails based on user preference. Lets call this function A
2) Custom content made in a module . Lets call this function B
3) Social integration . Lets call this function C
So what I did is as Function A is only called once a week I made a separate module for it.
As for Function B and C I put it all together as they would always be called together.
If you have problems with performance then check out this link . Its a good resource for performance improvement.
http://www.vmirgorod.name/10/11/5/tuning-drupal-performance
It lists a nice module called boost. I have not used it but I have heard good things about it.
cheers,
Vishal
Drupal .module files are all loaded with every page load. There is very little performance related that can be gained or lost simply by separating functions into different .module files. If you are not using an op code cache, then you can get improved performance by creating .inc files and referencing those files in the menu items in hook_menu, such that those files only get loaded when the menu items are accessed. Then infrequently called functions do not take up memory space.
File separation in general is a very small performance issue compared to how the module is designed with respect to caching, memory use, and/or database access and structure. Let the code maintenance and dependency issues drive when you do or do not create separate modules.
i'm actually interested in:
Is it better to create a single large custom module file for the sake of reducing database queries or break it into different smaller ones?
I was poking around and found a few things regarding benchmarking the database. Maybe the suggestion here is to fire up the dev version and test. check out db benchmarking
now i understand that doesn't answer specifically but i'd have to say its unique to each environment. I hate to use that type of answer but i truly believe it is. Depends on modules installed, versions used, hardware and os tunables among many other things.
A simple billing system (on top of ColdBox MVC) is ballooning into a semi-enterprisey inventory + provisioning + issue-tracking + profit tracking app. They seem to be doing their own thing yet they share many things including a common pool of Clients and Staff (login's), and other intermingled data & business logic.
How do you keep such system modular? from a maintenance, testability & re-usability stand point?
single monolithic app? (i.e. new package for the base app)
ColdBox module? not sure how to make it 'installable' and what benefits does it bring yet.
Java Portlet? no idea, just thinking outside the box
SOA architecture? through webservice API calls?
Any idea and/or experience you'd like to share?
I would recommend you break the app into modular pieces using ColdBox Modules. You can also investigate on separate business logic into a RESTful ColdBox layer also and joining the system that way also. Again, it all depends on your requirements and needs at the moment.
Modules are designed to break monolithic applications into more manageable parts that can be standalone or coupled together.
Stop thinking about technology (e.g. Java Portals, ColdBox modules, etc...) and focus on architecture. By this I mean imagining how you can explain your system to an observer. Start by drawing a set of boxes on a whiteboard that represent each piece - inventory, clients, issue tracking, etc... - and then use lines to show interactions between those systems. This focuses you on a separation of concerns, that is grouping together like functionality. To start don't worry about the UI, instead focus on algorithms and data.
If you we're talking about MVC, that step is focusing on the model. With that activity complete comes the hard part, modifying code to conform to that diagram (i.e the model). To really understand what this model should look like I suggest reading Domain Driven Design by Eric Evans. The goal is arriving at a model whose relationships are manageable via dependency injection. Presumably this leaves you with a set of high level CFCs - services if you will - with underlying business entities and persistence management. Their relationships are best managed by some sort of bean container / service locator, of which I believe ColdBox has its own, another example is ColdSpring.
The upshot of this effort is a model that's unit testable. Independent of of the user interface. If all of this is confusing I'd suggest taking a look at Working Effectively with Legacy Code for some ideas on how to make this transition.
Once you have this in place it's now possible to think about a controller (e.g. ColdBox) and linking the model to views through it. However, study whatever controller carefully and choose it because of some capability it brings to the table that your application needs (caching is an example that comes to mind). Your views will likely need to be reimagined as well to interact with this new design, but what you should have is a system where the algorithms are now divorced from the UI, making the views' job easy.
Realistically, the way you tackle this problem is iteratively. Find one system that can easily be teased out in the fashion I describe, get it under unit tests, validate with people as well, and continue to the next system. While a tedious process, I can assure it's much less work than trying to rewrite everything, which invites disaster unless you have a very good set of automated validation ahead of time.
Update
To reiterate, the tech is not going to solve your problem. Continued iteration toward more cohesive objects will.
Now as far as coupled data, with an ORM you've made a tradeoff, and monolithic systems do have their benefits. Another approach would be giving one stateful entity a reference to another's service object via DI, such that you retrieve it through that. This would enable you to mock it for the purpose of unit testing and replace it with a similar service object and corresponding entity to facilitate reuse in other contexts.
In terms of solving business problems (e.g. accounting) reuse is an emergent property where you write multiple systems that do roughly the same thing and then figure out how to generalize. Rarely if ever in my experience do you start out writing something to solve some business problem that becomes a reusable component.
I'd suggest you invest some time in looking at Modules. It will help with partitioning your code into logical features whilst retaining the integration with the Model.
Being ColdBox there is loads of doc's and examples...
http://wiki.coldbox.org/wiki/Modules.cfm
http://experts.adobeconnect.com/p21086674/
You need to get rid of the MVC and replace it with an SOA architecture that way the only thing joining the two halves are the service requests.
So on the server side you have the DAO and FACADE layers. And the client side can be an MVC or what ever architecture you want to use sitting somewhere else. You can even have an individual client for each distinct business.
Even for the server side you can break the project down into multiple servers: what's common between all businesses and then what's distinct between all of them.
The problem we're facing here luckily isn't unique.
The issue here seems not to be the code itself, or how to break it apart, but rather to understand that you're now into ERP design and development.
Knowing how best to develop and grow an ERP which manages the details of this organization in a logical manner is the deeper question I think you're trying to get at. The design and architecture itself of how to code from this flows from an understanding of the core functional areas you need.
Luckily we can study some existing ERP systems you can get a hold of to see how they tackled some of the problems. There's a few good open source ERP's, and what brought this tip to my mind is a full cycle install of SAP Business One I oversaw (a small-mid size ERP that bypasses the challenges of the big SAP).
What you're looking for is seeing how others are solving the same ERP architecture you're facing. At the very least you'll get an idea of the tradeoffs between modularization, where to draw the line between modules and why.
Typically an ERP system handles everything from the quote, to production (if required), to billing, shipping, and the resulting accounting work all the way through out.
ERPS handle two main worlds:
Production of goods
Delivery of service
Some businesses are widget factories, others are service businesses. A full featured out of the box ERP will have one continuous chain/lifecycle of an "order" which gets serviced by a number of steps.
If we read a rough list of the steps an ERP can cover, you'll see the ones that apply to you. Those are probably the modules you have or should be breaking your app into. Imagine the following steps where each is a different document, all connected to the previous one in the chain.
Lead Generation --> Sales Opportunities
Sales Opportunities --> Quote/Estimate
Quote Estimate --> Sales Order
Sales Order --> Production Order (Build it, or schedule someone to do the work)
Production order --> Purchase orders (Order required materials or specialists to arrive when needed)
Production Order --> Production Scheduling (What will be built, when, or Who will get this done, when?)
Production Schedule --> Produce! (Do the work)
Produced Service/Good --> Inventory Adjustments - Convert any raw inventory to finished goods if needed, or get it ready to ship
Finished Good/Service --> Packing Slip
Packing Slip items --> Invoice
Where system integrators come in is using the steps required, and skipping over the ones that aren't used. This leads to one thing for your growing app:
Get a solid data security strategy in place. Make sure you're confortable that everyone can only see what they should. Assuming that is in place, it's a good idea to break apart the app into it's major sections. Modules are our friends. The order to break them up in, however, will likely have a larger effect on what you do than anything.
See which sections are general, (reporting, etc) that could be re-used between multiple apps, and which are more specialized to the application itself. The features that are tied to the application itself will likely be more tightly coupled already and you may have to work around that.
For an ERP, I have always preferred a transactional "core" module, which all the other transaction providers (billing pushing the process along once it is defined).
When I converted a Lotus Notes ERP from the 90's to the SAP ERP, the Lotus Notes app was excellent, it handled everything as it should. THere were some mini-apps built on the side that weren't integrated as modules which was the main reason to get rid of it.
If you re-wrote the app today, with today's requirements, how would you have done it differently? See if there's any major differences from what you have. Let the app fight for your attention to decide what needs overhauling / modularization first. ColdBox is wonderful for modularization, whether you're using plugin type modules or just using well separated code you won't go wrong with it, it's just a function of developer time and money available to get it done.
The first modules I'd build / automate unit testing on are the most complex programatically. Chances are if you're a decent dev, you don't need end to end unit testing as of yesterday. Start with the most complex, move onto the core parts of the app, and then spread into any other areas that may keep you up at night.
Hope that helped! Share what you end up doing if you don't mind, if anything I mentioned needs further explanation hit me up on here or twitter :)
#JasPanesar