The Unexamined Code Is Not Worth Writing

– blogs from Harry Chou

In the Meantime, Paul and Mark Are Pair-Programming

leave a comment »

Written by harrychou

August 12, 2016 at 9:41 pm

Posted in Uncategorized

To Make Mark and Paul Happy …

leave a comment »

I’ve decided to write some new posts.

Here is the first one.

Let’s see when they would notice.

 

Written by harrychou

August 10, 2016 at 10:36 pm

Posted in Uncategorized

Note: Micro-Service Architecture, by Fred George

leave a comment »

==========================================================

My Summary:

  • Even experienced software designer/architect like Fred had his ideas rejected many times
  • Even rejected, he still push his idea of pub/sub and separated db for each service
  • Even only briefly mentioned, he used the word ‘bus’ once (and that’s the only time transport layer is mentioned)
  • So, prepare to receive rejections, and count it as blessing that even though rejected, we still get paid quite well
  • And, there are many ways to implement an idea, don’t be too obsessed with the choice of tools/framework
  • No super manager service that does orchestration …

==========================================================

2004 Project

– 1 M line of code, J2EE
– Largest Certified
– Started well, Martin Fowler
– Ideas becomes, PoEAA
– 70% success on Acceptance Test was ‘acceptable’
* different 30% of failure randomly
– Bug database has over 1000 entries
– still being changed
– new tests non-exist

Use Unit Test counts as metrics for project
– Unit test counts
– 40 developers writing 80 tests per week
* not very good
– try to write a new simple unit test takes a long time
* too much dependencies going on

So, a good project ended up becomes cumbersome.

Looking back, it is not 1M LineOfCode, but 100K LoC trying to get out
(only 10% of the code is useful, if rewrite, can take out 90% of code)

Why? What Happened?
– Lazy?
– Sloppy?
– Inexperienced! (send over to India)
– No power to refuse from developer!!
– Technical Debt? just a mask to the underlying problems …

2005 SOA in a Chinese Bank
– Bank services on multiple platforms
– basic idea: pub/sub
* events ‘published’
* interested application ‘subscribe’

– But, the idea was rejected, ended up the client bought a system overseas

Looking back, it is not 1M LineOfCode, not 100K LoC trying to get out,
but 20 5k LoC trying to get out
(break it down to micro services)

2005 Medical Systems
– rewrite platform to service hospital
– recommendation: pub/sub
– publish conclusions, not raw data
– example
* from Cat Scan
* about John Doe
* Time …
* Urgency: (Summary of information …)
* Validity: (Only relevant until …)

2005 Services for Investment Management
– the clients are successful
– prototype the new system architecture
– Bayesian Service Principles (Jeff Bay)
* It’s Okay to run more than one version at the same time
* You can only deploy one service at a time
– applying the principles can break dependencies …
(decoupled system)
– frequent deployments

2006 Batch Processing Replacement Orders
– pinball model
– use information packets bouncing around
– new observations
* services are like classes
– small crisp conceptualization
– services got very tiny (100 loc)
* smalltalk way of sending message passing
– send messages, not calling functions
* no database table sharing between services
(information hiding)
– challenges
* WTF is going on?
– cycles, lost packets
– needed to redefine, tracking, logging
* Inexperienced team in defining services
* approach compromised in Release 2
* people thought it was a bad idea ….

Looking back, it is not 1M LineOfCode, not 100K LoC trying to get out,
not 20 5k LoC, but 200 500 Loc trying to get out

2011 Current Company
– Monitor AdWords Accounts
* pub/sub system based on Linda Spaces
– separate databases to services
– define agent services for each other
– start to automate activities and recommendations
* off-shore team for implementation
* CTO killed it (he is Oracle guy, likes big database)
– replaced with traditional SQL-base solution
– CardWall
* front-end automated Card Wall in Agile
* back-end emerged as a set of services analyzing data
post alters
– delivered ….

New Observations
– Services became disposable (get rid of it if not used)
– Loosely coupled via RESTful Json packets or DB
* still knowledgeable about flow
– Self-monitoring services replaces Unit Tests
– Business-monitoring replaces Acceptance Tests
– Services are language-agnostics (programmers are excited about this)

Current State
– USwitch
* Design: Events, not Entities
* Events are more important than entities
* I the Plex: record everything
* Design: History not Current
* data is immutable
* keep both facts
– Legacy System
* Web/Services/Biz/Data/ETL/DataReporting
– Cloud of Signals
– Data Ecosystem
– discovery
* cross-selling tracking
* what pages are getting hit hard
* a lot of agile practices are not needed
– Technical Debt
* reduced because services are easy to understand and change

Conclusions:
– very very small
– loosely coupled
– multiple versions acceptable
– self-executing monitoring of each service
– publish interesting ‘stuff’ (even without requirements)
– application seems to be poor conceptualization

Living software systems
– long live systems, short-lived services (human body)
– extremely dynamic with continuous deployments
* 5-10 minutes between deployments typical
– Accept it is complex (especially for testing)
* acceptance test on business outcome instead
– radical impact to processes (Anarchy)
– there wll be a learning curve for developers

Theory of corporations
– will be in business forever

Theory of software
– assume will live forever

Written by harrychou

November 29, 2012 at 10:28 am

Posted in Uncategorized

RESTful Routes

leave a comment »

Ruby on Rails RESTful routes URL mapping (from http://ruby.railstutorial.org/chapters/sign-up#table-RESTful_users)

HTTP request URI Action Named route Purpose
GET /users index users_path page to list all users
GET /users/1 show user_path(user) page to show user
GET /users/new new new_user_path page to make a new user (signup)
POST /users create users_path create a new user
GET /users/1/edit edit edit_user_path(user) page to edit user with id 1
PUT /users/1 update user_path(user) update user
DELETE /users/1 destroy user_path(user) delete user

ASP.NET Web API Route Config

    public static class WebApiConfig
    {
        public static void Register(HttpConfiguration config)
        {
            config.Routes.MapHttpRoute(
                name: "DefaultApi",
                routeTemplate: "api/{controller}/{id}",
                defaults: new { id = RouteParameter.Optional }
            );
        }
    }

from http://www.asp.net/web-api/overview/getting-started-with-aspnet-web-api/tutorial-your-first-web-api

Controller Method URI
GetAllProducts /api/products
GetProductById /api/products/id
GetProductsByCategory /api/products/?category=category

Written by harrychou

September 7, 2012 at 9:17 am

Posted in Uncategorized

nServiceBus and MSMQ good info

leave a comment »

Let me have a place to store these useful links:

MSMQ FAQ (http://download.microsoft.com/download/F/C/9/FC9989A2-DA75-4D96-B654-4BD29CF6AEE1/MSMQ_faq_updated.doc)

How Applications send message to remote queue? (http://blogs.msdn.com/b/johnbreakwell/archive/2008/11/12/how-do-i-create-an-msmq-outgoing-queue.aspx#9131944)

  • do I need to create an outgoing queue?
  • what’s the behavior of the outgoing queue?
  • what’s the behavior of the outgoing queue under transaction?
  • how nServicebus implement fault tolerance?

 

Written by harrychou

August 24, 2012 at 8:38 am

Posted in Uncategorized

TDD Simple ASP.NET MVC3 Application with Entity Framework Code First and SQL Compact – Part 1

leave a comment »

This article is another ‘Ruby envy’ article. In this article, I tried my best to create a simple MVC3 application in the way that mimic, as much as I can, how Rails apps are created.

Here is a list of things I would like to achieve in this simple MVC3 application:

  • TDD – I would like to practice TDD in the way that I will write the test first, and let the test drive how the design will evolve. Don’t get your hope too high, this is a simple app and using TDD here is more just for the reason of doing it.
  • SQL Compact – One of the challenging areas of TDD is the DB access. I am using SQL Compact because it requires minimum setup and also works perfect with small set of data holding in memory.
  • Code First – One of the really awesome feature of Rails development is the seamless integration with database. I’d like to try and see how much Code First can make the process of TDD smoother.

The application is the traditional Blog Engine application. The idea is well understood by every developer and it demonstrates the essence of CRUD operations.

OK, hold on tight and we are going to get started in next post …

Written by harrychou

January 30, 2012 at 11:13 pm

Posted in Uncategorized

Remote Procedure Invocation/Call Style Architecture

leave a comment »

You’ve seen people use it all the time. There is nothing wrong with it. You call a remote service/routine like you would do to call a local object’s method and you get back the result. You complete your work and you deliver the system.

According to the page, there are disadvantages of RPC style architecture (with my updates and remark):

  • The major disadvantage is that there’s little room for code reuse. This is because the code for marshaling and unmarshaling, and the code for network communication is buried in the client and server applications. (Generally solved by standard based interoperability)
  • RPC isn’t language independent, and the client and the server must employ the same programming language. (See above)
  • There’s also very tight coupling between the applications. The reason is that because the calls are synchronous, the client application must wait for the server to complete the procedure before it can proceed further. Also, it’s possible to overcome this problem by using multithreading; however, it introduces another level of programming complexity that has inherent risks in terms of garbage collection.
  • The integration of the client and server is point to point and, therefore, isn’t suitable when a number of applications need to be integrated.
  • RPC is also not suitable if a large number of remote calls are involved. This is because of the synchronous nature of the call, which doesn’t allow the client to proceed before the server completes its work .
  • (RPC pretends to be just like local calls, which can fool developers if they ignore the differences in performance and availability)
There are advantages too (mine):
  • RPC encapsulate the functionalities of legacy systems and existing complex logic
  • RPC call is very easy to make – just like other procedure calls (which is a leaky abstraction and should be a disadvantage)

Written by harrychou

January 21, 2012 at 1:10 pm

Follow

Get every new post delivered to your Inbox.