quinta-feira, 21 de julho de 2016

Clean Code Summary and Key Points

A long time ago I used this summary of some key points that  I made to study about the book clean code. I hope it helps others.
This summary doesn't exclude the need to read the book Clean Code.


What is Clean Code?

The code can be measured with good or bad in the code review from how many WTF / or how many minutes you talk.
A clean code should be elegant, efficient, readable, simple, without duplications, well written, should be written carefully and must always have a responsibility.
You should add value to the business.
Clean code offers quality and understanding when we open a class.
It is necessary that your code is clean and readable for anyone to find it, can easily understand it avoiding waste of time and facilitating the work of all.

Meaningful Names:

Names of the classes, variables, methods, must be meaningful, clearly indicating that a method does or what an attribute is.
Should create pronounceable names to facilitate communication.
We should avoid acronyms.
Avoid confusing names, which can take anyone who reads the code to wrong conclusions.
Use names that reflect the system domain, the context and the problems that must be solved.

Functions:

The method should be easy to read and understand.
The method should convey its intention.
The methods should be small, another rule for small methods is that they should be even lower.
They must have up to 20 lines. (I think should have up to 10 lines)
Methods should only do one thing, they should do it the right way and just do it.
You should use names with words that can say what he really does.
The optimal number of parameters of a method is zero, after one and two.
Three should be avoided, but if it should have a good justification.
Parameters of the Boolean type as a parameter already clearly states that it does more than one thing.
Methods must do something and returns something.
Avoid duplication.

Comments:

One of the most common reasons for the comments is because the code is bad.
When thinking about writing a comment it is because the code should be refactored.
Comments do not save a bad code.
Try to explain what the code causes the code.
Comments can be useful when placed in certain places.
Create method names and informative variables instead of explaining the code with comments.
Use a comments can be used to express the importance of certain points in the code.
The best comment is one that needs to be written, because your code already explained.
Do not write comments redundant, useless or false information.
It should not be used to indicate who changed or why, for that already exist versioning.
Don’t comment code that will not be used, remove, it just pollutes the code and leaves no doubt in anyone reading.

Formatting:

It should give importance formatting since it is a developer of communication form.
A messy code is hard to read
The readability of the code will take effect on all the changes that will be made.
Try to write class with a maximum of 500 lines, smaller classes are easier to understand.
Set a limit of characters per line of code.
A good character limit on a line is 120.
Try to keep more next related concepts vertically to create a code stream
Use spaces between operators, parameters and commas.

Objects and Data Structure:

Follow the Law of Demeter:
One 'M' method of an object 'O' can only consume services of the following types of objects:
  • The object itself 'O;
  • The 'M' parameters;
  • Any object created / instantiated 'M';
  • Direct Components 'O'.
Make good abstraction and encapsulation.
Do not make dumb objects.
Objects hide the data abstraction and exposes methods that operate the data.
Data structure, exposes your data and do not have significant methods

Error Handling:

Error handling should be planned carefully by all programmers.
When happen wrong things we have to get him to do the right things
Should give preference to launch an exception than treat it just to hide.
Create message of some kind of information.
Mention it failed where was this failure, if possible why it failed.
Look separate business rules for errors or error handling.
Avoid returning a NULL in methods, preferably to return an empty object.
Avoid passing null to the methods, it can generate NullPointerExceptions.

Boundary:

In third-party code to avoid passing objects API’s looking forward to keep it in the same class.
Performs tests on API's Third Party
Study the documentation and well test the third API before you start using it.
Check well the features you will use
These step can help increase yield when there are new updates API and you can only run your tests to check for this update
Create tests the functionality of the API.

Unit Tests:
Make sure each piece of code is doing what you expect it to do.
Follow the TDD’s law: 
  • Not Create code before you have a failing test.
  • Shouldn’t create more tests than necessary to fail. 
  • You cannot write more code than enough to pass the test that is failing
Keep your test clean
The tests must undergo changes in the same way that the code
The dirtier the more difficult test will be, to give maintenance.
Look create an assert for testing
Use the F.I.R.S.T rule for testing:
  • The test is FAST running
  • The test should be INDEPENDENT of other
  • REPEATABLE in various environments
  • SELF- VALIDATING
  • TIMELY 
The test is as important as the production code.

Classes:

By default java classes should start with the variables:
  • Static and constant public
  • Static and variable private
  • Instances and variables privates
  • Soon after come the functions
The class name should represent your responsibility
The class must have only one responsibility
To know the size of the class is ideal or we should not measure her responsibility
You should try to make a brief description of the class
The methods should be:
  • Small
  • And even lower
  • They must have only one responsibility 

Systems:

It is important to recognize and separate responsibilities of a system.
It should be separate and modularize the logic execution, allowing an independent strategy for solving application dependency
Is an element important to take care of dependency injection allowing objects only take care of the business of logic.
It is very difficult to create a system properly first, must be made available to the story, then refactor it and the expand system to continue implementing new stories.
To get to that point is necessary TDD, refactoring and clean code.
We must build POJOs-based logic through testing and evolve from simple to interconnect the various aspects necessary.

Emergence:

Rules given by Kent Beck to create good designs:
  • Run all tests: they verify that the system behaves as expected.
  • Eliminate duplication: for with duplicate code has additional work.
  • To express the intention of the programmer: use more expressive code can to Facilitate maintenance. Choose good names for functions, classes and tests shouldn’t be small and well written. 
  •  Minimize the number of classes and methods: following this pattern can ignore it if the classes are very small.
  • Apply all knowledge to improve the design during refactoring: greater cohesion, reducing coupling, separate responsibilities, reducing classes and methods, choose the best names.
Even applying it once you will not be able to have good software you need to do this for over and over again, to achieve continuous improvement.

Concurrence:

The concurrency is an aspect that may be present in the codes.
For uncoupling allows improving the yield and structure of an application.
The concurrency can improve response times and application efficiency.
You should consider the following ideas about the concurrence:
  • It injects a certain overload.
  • It can be complex to operate.
  • Errors caused by it can be difficult to reproduce
  • Usually requires design changes.
The concurrence problem is that different segments of an application may be following tangled multi-threading, which can cause unexpected problems in normal situations.
For concurrence reasons it is important that each class has a unique responsibility. 
Create Sections synchronized minimized. Run tests are often the best way to find any errors in the code.
However, it is difficult to do when there are concurrence tests 
A good way to test is to insert codes for testing in the middle of the implemented code.

Successive Refinement:

The code only work is not enough to have a good code.
Professionals who care only about the code that works cannot be considered professional.
We can not consider that we have no time to refactor to one code, since this code that was not taken care of today can become a problem after becoming a problem for the team, because no one will want to mess with it.
Try not to let the rot code, it is much cheaper to create a clean code than cleaning a rotten code as move in a tangle can be an arduous task.
The solution then comes down to maintain the cleanest code possible and as simple as possible without ever leaving the rot started.

JUnit: 

Look to cover tests each not every method but each code line.
No code is immune improvement, and each of us has a responsibility to make the code a little better than we found it.
Refactoring is an iterative process full of trial and error inevitably converging to something that we feel is worthy of a professional.

Refactoring:

Before making any kind of refactoring is important to have good coverage tests.
After increase or create test coverage can begin to leave the clearest code and fix some bugs.
Now after leaving the code clearer probably someone else can clean even more.

Smells and Heuristics:

What you should not do about comments:
  • Inappropriate information
  • Obsolete Comment
  • Redundant Comment
  • Review misspelled
  • Commented Code
Environment:
  • Build requires more than one step
  • Tests require more than one step
 Functions/Methods:
  • Many arguments
  • Flags with Boolean
  • Methods unused
General
  • Multiple Languages to a file.
  • Overridden Insurance
  • Code Duplication
  • Code on the wrong level of abstraction
  • Class basis depending on derivatives
  • Lots of information in a method
  • Dead Code (Unused)
  • Vertical Separation
  • Inconsistency
  • Disorder
  • Responsibility BLEND
  • Static inappropriate
  • Use explanatory variables
  • Function names should say what they do
  • Understand the Algorithm
  • Make logical dependencies
  • Make Polymorphism to instead of If / Else or Switch / Case
  • Follow standard conventions
  • Be precise
  • Encapsulating conditionals
  • Avoid negative conditionals
  • Functions should do something
  • Functions should come down only one level of abstraction
  • Maintain configurable data at high levels
Java
  • Avoid long lists of import using *
  • Not Inherit Constants
  • Prefer to instead of Enums Constants
Names
  • Choose descriptive names
  • Choose names at the appropriate level of abstraction
  • Use standard nomenclature where possible
  • Clear names
  • Use long names for long Scopes
  • Avoid encodings
  • Names should describe side effects
Tests
  • Lack of testing
  • Use a cover tool!
  • Make Trivial tests
  • A test is ignored a question about an ambiguity
  • Test coverage patterns can be revealing
  • Tests should be fast 
Clean Code is not written following a set of rules.
You do not become a software professional learning just a list of what you do and what n is done.

Professionalism and craftsmanship comes from values and discipline in lists of what you should and should not do when creating a code.

quarta-feira, 13 de julho de 2016

The best feature of Vagrant - Multi-Machine

Vagrant allows to be created a development environment that can be distributed. If you have a team that uses different operating systems, it is possible to assemble a virtual environment standard for all.

Vagrant can run with VirtualBox, VmWare.
It is virtually automate the creation of a virtual machine to the virtual box.

It can be considered a management tool for virtual machines
It is focused to creating environments that are similar or identical as possible to production servers.


Boxes that were possible use: Windows, Ubuntu, Debian, and CentOS

Providers: VirtualBox, VmWare, Amazon web services.


Provisioners:

Tools industry standard provisioning, such as shell scripts, Chef or Puppet, can be used to install and configure the software on the machine automatically.





Here are some reasons I've seen for using Vagrant over just VirtualBox.

1. Set Up Multi-VM Networks with Ease

Most of the Vagrant power-user content I've read has been about setting up multiple VMs at the same time. Vagrant gives you a single config file to set these up, enabling you to launch all of them with one command.

Say you've configured three VMs to network with each other using static IPs on the 192.168.1.* subnet. You find yourself in a location that is already using that subnet to hand out IP addresses, and your VMs now conflict. With Vagrant, you can simply edit the Vagrantfile and reload the VMs, whereas with VirtualBox you'd have to open the settings for each VM, if not boot each VM and change them inside.

2. Source Control

By putting the settings in a text file, it enables the configuration to be put under source control. Made some changes last week and accidentally broke the image? Just revert the changes and reload the VM. You can accomplish this with VirtualBox snapshots, but it will take up much more space than just a Vagrantfile.

3. Various Platforms
There are a large number of boxes available at sites such as http://vagrantbox.es. This enables you to try various OS or distributions, applying the same provisioning to set up similar environments. This can help with testing or adding support to new platforms, and would be time-consuming using just VirtualBox.

Why:

Vagrant allows you to create portable work environments, which can be easily reproduced on any system. It's useful for developers and system administrators who want to mimic a server's configuration on their local machine. It's useful for teams who want to quickly get designers and project managers up and running with a local installation.

Vagrant is a wrapper, which ties together several components including: virtualization software, such as VirtualBox; a server base box, such as Ubuntu provisioning tools for configuration management, such as Chef.

 SETUP

The first step in configuring any Vagrant project is to create a Vagrantfile. The purpose of the Vagrantfile is two folds: 
  •  Mark the root directory of your project. A lot of the configuration of Vagrant is relative to this root directory.
  •  Describe the kind of machine and resources you need to run your project, as well as what software to install and how you want to access it.


The Vagrantfile is meant to be committed to version control with your project, if you use version control. This way, every person working with that project can benefit from Vagrant without any upfront work.

VagrantFile Sample:


Puppet Sample:


Multi-Machine:

Vagrant is able define and control multiple guest machine per VagrantFile.
These machines are generally able to work together or are somehow associate with each other.
Accurately modeling a multi-server production topology, such as separating a web and database server.
Modeling a distributed system and how they interact with each other.
Testing an interface, such as an API to service component.

VagrantFile to multi-machine sample:

 
To access machines: 

1. vagrant up
2. vagrant ssh n1
3. vagrant ssh n2






References:



sexta-feira, 1 de julho de 2016

How works Network and API to VirtualBox


VirtualBox is a high-performance, cross-platform virtualization engine for use on computers running Windows, Linux distributions and others. Designed for use on Intel and AMD x86 systems, VM VirtualBox can be deployed on desktop or server hardware. As a hosted hypervisor, it extends the existing operating system installed on the hardware rather than replacing it.

VirtualBox includes a hypervisor for the host platform, an application programming interface (API) and software development kit (SDK) for managing guest virtual machines, a command-line tool for managing guests locally, a web service for remote management of guests, a wizard-style graphical tool to manage guests, a graphical console for displaying guest applications on the local host, and a built-in Remote Desktop Protocol (RDP) server that provides complete access to a guest from a remote client.

VirtualBox Architecture






VirtualBox API = VBoxMananger = CLI

VBoxMananger is the command line interface to VirtualBox, if put command VBoxMananger in the console, is possible see all commands to interact with VirtualBox API.



Command to list all VMs:


Is Possible creating VMs with command line, but is necessary have ISO to system operation that want install.

Step 1:


Step 2:


VBoxMananger is the tool that Vagrant use for sample to create environments.

VirtualBox Network

There are 3 types of network:

  1. Nat with port forwarding
  2. Bridged networking
  3. Internal networking
  4. Host-only networking


Nat Networking:

  • Nat = Networking Address Translation
  • Each VM has your own sub-network with your public IP
  • Share the same resources that Host
  • VMs cannot see each other
  • IP, Gateway, DNS cannot be change


Bridged Networking:

  • VMs and Host are in the same network
  • VMs has access to physical network in the same way that Host
  • Can be found in the physical network


Internal Networking:

  • Isolated network
  • Only VMs are within a private network
  • Host can’t see the private network


Host-Only Networking:

  • Hybrid between bridge and internal network
  • Private Network to Host and VMs
  • VMs can’t see external network
  • Host has private IP to internal network

My Slides

In this post I put focus just on point that is difficult find people talk about (VirtualBox API and VirtualBox Network) and some times ago I needed study, so I tried put here somethings that was useful for me.

References:


quinta-feira, 23 de junho de 2016

How do Data Ingestion with Apache Flume sending events to Apache Kafka

What is data ingestion?

Date ingestion is the process of getting data from somewhere like logging, database for immediate use or for storage.

These data could be sent in real time or in Batches.
Ingestion by Real time data is sent one by one directly from the data source, as in batch mode data are taken in batches in pre-defined intervals.



Usually when ingestion date of talks is likely to be related to Big Data, a large volume of data and assigned to it a few concerns us may come in mind, such as volume, variety, velocity and Veracity.
For this work Data Ingestion, a consolidated tool and widely used is Apache Flume.

Apache Flume 

Apache flume is an open source tool, reliable, high-availability for aggregation, collection and able to move large amounts of data from many different sources into a centralized data storage.
Based on data flow streaming is robust, fault-tolerant with high reliability mechanisms.

A Flume event is defined as a data flow unit. A Flume agent is a process of the JVM that hosts the components through which events flow from an external source to the next destination.

 The idea in this post is to represent the flow data model below focusing on the necessary settings for this flow happen, abstracting some concepts that should be checked in the flume if necessary documentation. 


Tools required to test the sample:



Configuration files:

After downloading the Apache flume should be created a configuration file in the folder conf/ "flume-sample.conf"

This file can basically be divided for better understanding of 6 parts: 

1.Agent Name:
a1.sources = r1
a1.sinks = sample 
a1.channels = sample-channel

2.Source configuration:
a1.sources.r1.type = exec
a1.sources.r1.command = tail -f /log-sample/my_log_file.log  
a1.sources.r1.logStdErr = true

3.Sink type
a1.sinks.sample.type = logger

4.Buffers events in memory to channel
a1.channels.sample-channel.type = memory
a1.channels.sample-channel.capacity = 1000
a1.channels.sample-channel.transactionCapacity = 100

5. Bind the source and sink to the channel
a1.sources.r1.channels.selector.type = replicating
a1.sources.r1.channels = sample-channel

6.Related settings Kafka, topic, and host channel where it set the source 
a1.sinks.sample.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.sample.topic = sample_topic
a1.sinks.sample.brokerList = 127.0.0.1:9092
a1.sinks.sample.requiredAcks = 1
a1.sinks.sample.batchSize = 20
a1.sinks.sample.channel = sample-channel

Final result of the file below:



To run Apache Flume with this configuration you must run the following command within the Flume folder:

sh bin/flume-ng agent --conf conf --conf-file conf/flume-sample.conf  -Dflume.root.logger=DEBUG,console --name a1 -Xmx512m -Xms256m

Where :
  • Indicate where the file that was set flume-sample.conf
  • "- - Name" is the agent name equals to a1
  • Dflume.root.logger is the form that will be logged on the console 


Before it is necessary to raise the Apache Kafka (concepts related to Kafka are not part of this post the focus here is only the data ingestion with flume)

After downloading the Kafta with the default settings, you can see the flow work.
 
Execute the following commands: 


1. START Zookepper -  sudo bin/zookeeper-server-start.sh config/zookeeper.properties&
2. START Kafka Server - sudo bin/kafka-server-start.sh config/server.properties&
3. CREATE TOPIC - bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --create --replication-factor 1 --partitions 1 --topic sample_topic

In this command should show the data being extracted from the Source (my_log_file.log) and are being sent to the sample_topic topic.

4. START COMSUMER - bin/kafka-console-consumer.sh --zookeeper 127.0.0.1:2181 --topic sample_topic --from-beginning


Thus the proposed data flow is complete and providing data to other consumers in real time.


References:

sexta-feira, 17 de junho de 2016

Understanding Reactor Pattern - Thread-based and Event-driven

To handle web requests, there are two competitive web architectures thread-based one and event-driven one.

Thread-based Architecture

The most intuitive way to implement a multi-threaded server is to follow the process/thread-per-connection approach.

It is appropriate for sites that need to avoid threading for compatibility with non-thread-safe libraries. 

It is also the best Multi-Processing Modules for isolating each request, so that a problem with a single request will not affect any other.


Processes are too heavyweight with slower context switching and memory-consuming. Therefore, the thread-per-connection approach comes into being for better scalability, though programming with threads is error-prone and hard-to-debug.

In order to tune the number of threads for the best overall performance and avoid thread-creating/destroying overhead, it is a common practice to put a single dispatcher thread  in front of a bounded blocking queue and a thread pool. The dispatcher blocks on the socket for new connections and offers them to the bounded blocking queue. Connections exceeding the limitation of the queue will be dropped, but latencies for accepted connections become predictable. A pool of threads polls the queue for incoming requests, and then process and respond.

Unfortunately, there is always a one-to-one relationship between connections and threads. Long-living connections like Keep-Alive connections give rise to a large number of worker threads waiting in the idle state for whatever it is slow, e.g. file system access, network, etc. In addition, hundreds or even thousands of concurrent threads can waste a great deal of stack space in the memory.

 Event-driven Architecture

Event-driven approach can separate threads from connections, which only uses threads for events on specific callbacks/handlers.

An event-driven architecture consists of event creators and event consumers. The creator, which is the source of the event, only knows that the event has occurred. Consumers are entities that need to know the event has occurred. They may be involved in processing the event or they may simply be affected by the event.

The Reactor Pattern

The reactor pattern is one implementation technique of the event-driven architecture. In simple words, it uses a single threaded event loop blocking on resources emitting events and dispatches them to corresponding handlers/callbacks.

There is no need to block on I/O, as long as handlers/callbacks for events are registered to take care of them. Events are like incoming a new connection, ready for read, ready for write, etc.

Those handlers/callbacks may utilize a thread pool in multi-core environments.

This pattern decouples modular application-level code from reusable reactor implementation.

There are two important participants in the architecture of Reactor Pattern:

1. Reactor

A Reactor runs in a separate thread and its job is to react to IO events by dispatching the work to the appropriate handler. It’s like a telephone operator in a company who answers the calls from clients and transfers the communication line to the appropriate receiver.

2. Handlers

A Handler performs the actual work to be done with an I/O event similar to the actual officer in the company the client who called wants to speak to.


Reactor responds to I/O events by dispatching the appropriate handler. Handlers perform non-blocking actions.

The intent of the Reactor pattern is: 

The Reactor architectural pattern allows event-driven applications to demultiplex and dispatch service request that are delivered to an application from on or more clients.

One Reactor will keep looking for events and will inform the corresponding event handler to handle it once the event gets triggered.

 The Reactor Pattern is a design pattern for synchronous demultiplexing and order of events as they arrive.

It receives messages/requests/connections coming from multiple concurrent clients and processes these post sequentially using event handlers.

The purpose of the Reactor design pattern is to avoid the common problem of creating a thread for each message/request/connection.



In Summary: Servers has to handle more than 10,000 concurrent clients and Threads can not scale the connections using Tomcat /Glassfish/ Jboss /HttpClient.

Then receives events from a set of handles and distributes them sequentially to the corresponding event handlers.

So, the application using the reactor only needs to use a thread to handle events simultaneously arriving.



Basically the standard Reactor allows a lead application with simultaneous events, while maintaining the simplicity of single threading.

A demultiplexer is a circuit that has an input and more than one output.
It is a circuit used when you want to send a signal to one of several devices.
This description sounds similar to the description given to a decoder, a decoder, but is used to select between many devices while a demultiplexer is used to send a signal, among many devices.




A Reactor allows multiple tasks which block to be processed efficiently using a single thread.
Reactor manages a set of event handlers and executes a cycle.

When I called to perform a task that connects with a new or available handler becoming active.
When called to perform a task, it connects with the handler that is available and makes it as active.

The cycle of events:

1 - Finds all handlers that are active and unlocked or delegates this for a dispatcher implementation.

2 - Execute each of these handlers sequentially found until complete or reach a point where they block.
Handlers completed inactivate or assets for reuse, allowing the event cycle to continue.

3 - Repeats from Step One (1)



Why matter now a day? 

Because the Reactor pattern is used by Node.js, Vert.x, Reactive Extensions ,Netty, Ngnix and others. So if you like identify pattern to know how thinks works behind the scenes, is important pay attention in this pattern.


References: