I have a codebase where one file contains quite a lot of Structs, Interfaces and Variables in the same file as functions and I'm not sure if I need to seperate this into separate files with appending filename. So for example accounts.go will be accounts_struct.go and accounts_interface.go with struct and interface respectively.
What would be a good approach for the file organisation when you have growing codebase for Structs, Variables and Interfaces?
A good model to check out is the source code of Go itself: http://golang.org/src
(in addition of the official "Effective Go")
You will see that this approach (separating based on language items like struct, interface, ...) is never used.
All the files are based on features, and it is best to use a proximity principle approach, where you can find in the same file the definition of what you are using.
Generally, those features are grouped in one file per package, except for large ones, where one package is composed of many files (net, net/http)
If you want to separate anything, separate the source (xxx.go) from the tests/benchmarks (xxx_test.go)
As Thomas Jay Rush adds in the comments
Sometimes source code is automatically generated -- especially data structure definitions.
If the data structures are in the same file as the hand-wrought code, one must build capacity to preserve the hand-wrought portion in the code-generation phase.
If the data structures are separated in a different file, then inclusion allows one to simply write the data structure file without worry.
Dave Cheney offers an interesting perspective in "Absolute Unit (Test) # LondonGophers" (March 2019)
You should take a broader view of the "unit" under test.
The units are not each internal function you write, but a whole package. Specifically the public API of a package.
Organizing your files to facilitate testing their Public API is a good idea.
accounts_struct_test.go would not, in that regards, make much sense.
See also "How I organize packages in Go" by Bartłomiej Klimczak
Sometimes, a few handlers or repositories are needed.
For example, some information can be stored in a database and then sent via an event to a different part of your platform. Keeping only one repository with a method like saveToDb() isn’t that handy at all.
All of elements like that are split by the functionality: repository_order.go or service_user.go.
If there are more than 3 types of the object, there are moved to a separate subfolder.
Here is my mental model for designing a package.
a. A package should encompass one idea or concept. http is a concept, http client or http message is not.
b. A file in a package should encompass a set of related types, a good rule of thumb is if two files share the same set of imports, merge them. Using the previous example, http/client.go, http/server.go are a good level of granularity
c. Don't do one file per type, that's not idiomatic Go.
Related
I have read the Go Tour and Googled "golang packages" but I have not yet found any advice about best practice in Go for organising moderately sized applications.
If I have an application that conceptually has several distinct parts, perhaps 10^3-10^4 LOC, and I don't intent to create reusable libraries for use in other applications, should all the source code files be package main?
To clarify ...
For example, lets say my program will have the following major chunks:
Something that manages a bunch of persistently stored data
allowing usual create, read, update, delete operations
Something that allows a human to view the stored data
Something that coordinates / mediates between these
Something that periodically fetches data updates from a web-service using SOAP.
So that would be MVC plus a fetcher of data.
From looking around at what people do, I now suspect I should
create $GOPATH/src/myprogramname
in there put some main.go with package main and func main() { ... } in it.
create some subdirectories like
$GOPATH/src/myprogramname/model
$GOPATH/src/myprogramname/view
$GOPATH/src/myprogramname/control
$GOPATH/src/myprogramname/fetch
have the .go files in those subdirectories begin with package fetch, etc. Where the package name always matches the subdirectory name.
my main.go will probably import ( ... "fetch"; "model"; "view"; "control" )
as main.go grows, split it into other reasonably sized .go files named according to purpose.
build the program, including *.go in the above package subdirectories by
cd $GOPATH/src/myprogramname
go build
Is that all I need to do? Is that the properly idiomatic Go way of organising things? Is there more I should know or be thinking of? Is there some canonical webpage or PDF I overlooked and should read to find out this stuff?
In short, I don't want a 10,000 line main.go with everything in it. What are the idiomatic Go principles for organising code into files, subdirectories, packages and any other organisational units corresponding to normal conceptual divisions according to well-known structured-programming and/or OO principles?
You could break down your project into several layers based on the encapsulation level of your functions, i.e. having low-level functions in separate packages and logic functions in your main package. (You could inspire yourself of MVC-like architectures)
Since we don't have any details about your code, it is hard to see what kind of architecture would be best suited.
But in the end your choice will be based on the code simplicity / re-usability balance.
The general "best practice" in Go seems to be having each package provide a type or a service. Most of the packages in the standard library expose one or two types, and functions for working with those types. Some, like net/http and testing, provide a service - not in the "microservices" sense of something executable in itself, but rather a set of functionality related to a specific activity.
I am currently working on a very complex program that processes rows from an input table and has a huge number of possible outcomes for each record. Because of this I have a very large number of constants defined for the outcome messages. There is one success message for the record, but a multitude of possible warnings and errors.
My first thought was to define all of my constants for these messages at the package body level, but then I decided to move each constant to the procedure where it is used. I'm now second guessing that decision and thinking of moving everything back to package body level. What is the best way to define this many constants? Ease of maintainability is my ultimate goal for this program since it is so complex.
I think this is a matter of taste. In my application I put all error codes into an Error-Package. All main and commonly used constants I put into a separate package (without a package body).
Again, a matter of taste, but I tend to put a list of named constants at the package spec level rather than the package body so that they can be referenced by any portion of the application. If I ever want to change the error code that c_err_for_specific_reason_x uses, it becomes a single place to do so.
If I wanted to hide the codes and put them within the body I would have a get_error_code(p_get_error_name varchar) function that did the translation based on you passing a valid constant name.
I've done both on different projects, but tend towards the list over the function most times. I tend to use the function if it a table-driven source of the data.
It ... wait for it ... depends!
Since you currently define your constants in the package body, you don't need them to be publicly accessible outside the package. So defining them in a spec really doesn't buy you anything.
Here's is the rule I follow: Define constants within the smallest scope needed. So if a constant is used only within one procedure, define it in that procedure. If it is used within more than one procedure, define it in the body. If it is used elsewhere by code in other packages (or non-packaged SPs) but only when using a particular package, define it in the spec of that package. If it is used by other code for general use, put it in a separate spec of such general constants.
I am designing a new YAML file, and I want to use the most standard style of naming. Which is it?
Hyphenated?
- job-name:
...
lower_case_with_underscores?
- job_name:
...
CamelCase?
- jobName:
...
Use the standard dictated by the surrounding software.
For example, in my current project the YAML file contains default values for Python attributes. Since the names used in YAML appear in the associated Python API, it is clear that on this particular project, the YAML names should obey the Python lower_case_with_underscores naming convention per PEP-8.
My next project might have a different prevailing naming convention, in which case I will use that in the associated YAML files.
Kubernetes using camelCase: https://kubernetes.io/docs/user-guide/jobs/
apiVersion, restartPolicy
CircleCI using snake_case: https://circleci.com/docs/1.0/configuration/
working_directory restore_cache, store_artifacts
Jenkins with dash-case: https://github.com/jenkinsci/yaml-project-plugin/blob/master/samples/google-cloud-storage/.jenkins.yaml
stapler-class
So it looks like projects and teams use their own conventions and there is no one definite standard.
A less popular opinion derived from years of experience:
TL;DR
Obviously stick to the convention but IMHO follow the one that is established in your project's YML files and not the one that comes with the dependencies. I dare to say naming convention depends on too many factors to give a definitive answer or even try to describe a good practice other than "have some".
Full answer
Libraries might change over time which leads to multiple naming conventions in one config more often than any sane programmer would like - you can't do much about it unless you want to introduce (and later maintain) a whole new abstraction layer dedicated to just that: keeping the parameter naming convention pristine.
A one example of why you would want a different naming convention in your configs vs. configs that came with the dependencies is searchability, e.g. if all dependencies use a parameter named request_id, naming yours request-id or requestId will make it distinct and easily searchable while not hurting how descriptive the name is.
Also, it sometimes makes sense to have multiple parameters with the same name nested in different namespaces. In that case it might be justified to invent a whole new naming convention based on some existing ones, e.g.:
order.request-id.format and
notification.request-id.format
While it probably isn't necessary for your IDE to differentiate between the two (as it's able to index parameters within the namespace) you might consider doing so anyway as a courtesy for your peers - not only other developers who could use different IDEs but especially DevOps and admins who usually do use less specialized tools during maintenance, migrations and deployment.
Finally, another good point raised by one of my colleagues is that distinctive parameter names can be easily converted into a different convention with something as simple as one awk command. Doing so the other way around is obviously possible but by an order of magnitude more complicated which often spawns debates in the KISS advocates community about what it really means to "keep it simple stupid".
The conclusion is: do what's most sensible to you and your team.
We're doing a big project on OSGi and adding some commons modules. There's some discussion about naming the artifact.
So, one possibility when naming the module is for example:
cmns-definitions (for common definitions), another is cmns-definition, still another is cmns-def. This has some effect also on the package name. Now it's
xx.xxx.xxx.xxx.xxx.commons.definitions, if changing to cmns-def it would be xx.xxx.xxx.xxx.xxx.commons.def.
Inside this package will be classes like enums and other definitions to be used throughout the system.
I personally lean to cmns-definitions since there's not only 1 definition inside the package. Other people point out that java.util doesn't have only 1 utility there for example. Still, java.util is an abbreviation for me. It can mean java utility or java utilities. Same thing happens with commons-lang.
How would you name the package? Why would you choose this name?
cmns-definitions
cmns-definition
cmns-def
Bonus question: How to name something like cmns-exceptions? That's how I name it. Would you name it cmns-xcpt?
ËDIT:
I'm throwing in my own thoughts on this in the hope of being either confirmed or contradicted. If you can, please do.
According to what I think, the background reason why you name something is to make it easier to understand what's inside it. Or, according to Peter Kriens, to make it easy to remember and being able to automate processes via patterns. Both are valid arguments.
My reasoning is as follows in terms of pattern:
1) When a substantivation occurs and it's well known in the industry, follow it on your naming.
Eg:
"features" is a case on this. We have a module called cmns-features. Does this mean we have many features on this module? No. It means "the module that implements the "features" file from Apache karaf".
"commons" is a substantivation of "common" well-accepted on the industry. It doesn't mean "many common". It means "Common code".
If I see extr-commons as a module name, I know that it contains common code for extr (in this case extraction), for example.
2) When a quantity of classes inside the module are cooperating to give a distinct "one and one only" meaning to the whole, use singular form to name it.
The majority of modules are included here. If I name something cmns-persistence-jpa, I mean that whatever classes inside cooperate together to provide the jpa implementation of cmns-persistence-api. I don't expect 2 implementations inside it, but actually a myriad of classes that together make one implementation. Crystal clear to me. No?
3) When a grouping of classes is done with the sole purpose of gathering classes by affinity, but the classes don't cooperate together to no purpose, use plural.
Here is the case for example of cmns-definitions (enums used by the whole system).
Alternatively, using an abbreviation circumvents the problem, e.g. cmns-def which can be also "interpreted expanded" by a human reader to cmns-definitions. Many people use also "xxxx-util" meaning xxxx-utilities.
Still a third option can be used to pack things together, using a name that itself means a pluralization. The word "api" comes to mind, but any word that pluralizes something would do, like "pack".
Support to these cases (3) are well-known modules like commons-collections (using the plural) or commons-dbcp (using abbreviation) or commons-lang (again abbreviation) and anything that uses api to pack classes together by affinity.
From apache:
commons-collections -> many powerful data structures that accelerate development of most significant Java applications
commons-lang -> host of helper utilities for the java.lang API
commons-dbcp -> package of several database connection pools
'it is just a name ...'
I find in my long career that these just names can make a tremendous difference in productivity. I do not think it makes a difference if you use definitions, definition, or def as long as you're consistent and use patterns in the name that are easy to remember and can be used to automate processes. A build based on a consistent naming scheme is infinitely easier to work with than a build with "nice human display" names that are ad-hoc and have no discernible pattern.
If you use patterns, names tend to become shorter. Now people working with these names usually spent a lot of time with them. So their readability is not nearly as important as their mnemonic value. It turns out that abbreviations of 3 or 4 characters are surprisingly powerful. One of the reason is they work well is that there is only one possible abbreviation while if you go longer there are many candidates.
Anyway, most import part is the overall consistency. Good luck.
definitions (or def or definition) is a bad name because it doesn't have any semantic to a reader. You're in an object oriented world (I suppose) - try to follow its conventions and principles. Modules in Maven should be named after the biggest "abstraction" they contain. "Definition" is a form, not a meaning.
Your question is similar to: "Which class name is better FileUtilities or FileUtils". Answer: none.
Basically what you do with the Definitions and Exceptions is to provide kind of an API for your other modules. So I propose to combine definitions, exceptions and add interfaces to it. Then it makes sense to call it all cmns-api. I normally prefer the singular names as they are shorter but you are free to decide as it is just a name.
I am writing a script that takes data from a database and creates GoogleChart URLs from the parsed data. I only need to create two type of charts, Pie and Bar, so is it wrong if I stick both of those classes in the same file just to keep the number of files I have low?
Thanks
If you're asking the "ruby" way, then it is to put your classes in separate files. As some others have alluded to, placing your classes in separate files scales better. If you place multiple classes in the same file and they start to grow, then you're going to need to separate them later.
So why not have them separate from the beginning?
UPDATE
I should also mention that autoload works by expecting classes to be in their own files. For instance, if you're in a Rails environment and you don't separate classes into different files, you'll have to require the file explicitly. The best place to do this would be in the application.rb file. I know you're not in a Rails environment, but for others who may find this answer, maybe this info will be helpful.
UPDATE2
By 'autoload', I meant Rails autoload. If you configure your own autoload, you can put classes in the same file; but again, why? The Ruby and Java communities usually expect classes to be in separate files. The only exception are nested classes, but that's for more advanced design patterns.
Usually more less-complex files are better than less more-complex ones. Specially if you need to share the code with others.
It's not wrong. If your code is simple enough* then by all means put all of it in one file.
On the other hand, if you think your code is going to get more complex later, or if you plan to use automated testing tools later, you will be doing yourself a big favour if you deal with the structure of all that now.
(* My personal rule of thumb: about 200 lines.)