Monday, May 9, 2016

Lambda Environment Configuration with DynamoDB and /tmp


Currently AWS does not have a mechanism for Lambda deployment to encapsulate environment specific configuration.  You must include configuration files alongside your Lambda functional code.
When working on a photo architecture (https://runsignup.wordpress.com/2016/03/29/runsignup-photo-architecture/) we require ability to be a bit more dynamic in our approach for configuration and not depend on code updates.  We decided on using DynamoDB for managing this configuration, and improving performance by taking advantage of the /tmp storage available in container re-use as described here.   Below are the options we reviewed and the code for using Lambda + Dynamo + /tmp storage.

Here are some options for being more dynamic and code example for the approach we used.

[Note: If you are using API Gateway there is a better workaround, read here, we require S3 triggers]

a.  Update Lambda every time configuration changes and redeploy. 

Having configuration inside Lambda is a good from a performance perspective, but managing many customer specific Lambda's would be problematic.

b.  Use the Lambda description to encapsulate config object.

A hack that works (confirmed) and detailed here, however there is a  256 character limit for description field per documentation.

c. Use S3 to store the configuration file.

This allows remote storage of configuration via a file.  What's better Dynamo as document or S3 as a file?  Not too much, though the folks at Concurrency Labs had the same thought process and did a some performance measurements.  You can read more about their insights at their blog post here, the following is from their results.

d. Use a more dynamic approach with DynamoDB

Our approach was to use DynamoDB for the configuration store, while using /tmp to gain performance benefit when the container is re-used. 



Monday, January 25, 2016

AWS Certificate Manager, Easy Button


I know the pains of managing SSL certificates for a data center or even the cloud.  It is a manageable pain, you can live with it, but no one wanted to deal with the pain.  

  • Acquiring an SSL Cert was never automated
  • Deploying SSL Cert to network gear had challenges
    • Automating between different vendors
    • Keeping scripts updated
    • Adding/Removing network gear
    • Documenting the process, so next person could do it
  • Calendar/Scripts to notify when CERTs are expiring
  • Project Manager reminding people to do the work
I recently built a RunSignUp racing results app using Ionic/PhoneGap/Angular which is built for Web, Android, and iOS.  One of the items on the to-do list - setup SSL cert.  I was dreading this task, the details of getting the moving parts correct is a time sink.  Well here is a case where slight procrastination worked out pretty well - AWS announced the AWS certificate manager. 


1. Add your domain name


2. Review your request


3. Wait for validation

We can see in status in list view as well





4. Site owner receives and responds to email


5. We are now ready to go!


6. Before the CERT was deployed


7. We can now make our cloudfront domain use the cert.


8. Send a Cache Invalidation

9. And now we are green and serving on SSL


10. Change CloudFront behavior to alway redirect to HTTPS






Wednesday, January 13, 2016

Gulp bump minor number for ionic

If you need a full blown approach for bumping your version numbers then there is a robust set of build tools by Robert Shedd 
https://github.com/shedd/ionic_build_tools#bump-app-versions
His build tools provide more than just gulp commands, the readme alone can save you time and help you avoid mistakes.

However, I just needed something simple for bumping version numbers in package.json and config.xml.

Extra NPM packages

  • npm install gulp-xml-editor --save-dev
  • npm install gulp-json-transform --save-dev 


Requires

  • var jsonTransform = require('gulp-json-transform');
  • var xeditor = require('gulp-xml-editor');

Gulp Task
gulp.task( 'package', ['sass'], function(){
gulp.src('./package.json')
.pipe( jsonTransform(function(data){
var version = data.version.split('.');
version[2]++;
data.version = version.join('.');
return data;
},2))
.pipe(gulp.dest('.'));
gulp.src("./config.xml")
.pipe(xeditor(function(xml, xmljs) {
var version = xml.root().attr('version').value().split('.');
version[2]++;
  xml.root().attr('version',version.join('.'));
  return xml;
}))
 .pipe(gulp.dest("."));
});





Thursday, January 7, 2016

Bucket List item Done - Firefox Plugin

It may have taken a decade and a good reason but I finally wrote a Firefox Plugin!

Why Firefox and not Chrome?  

It is an integration with Selenium IDE which is a Firefox extension.

What does it do?

Provided integration with Selenium IDE to record browser traffic and save results as JMeter load test (.jmx file).  You can turn it on, run your Selenium test, turn if off and then you have JMeter version of your Selenium test.  Take the output and load it directly into JMeter or run it scale on RedLine13.

Why did I write this?

A use case I hear often is how to scale real user testing.  The typical answer is run some Selenium tests in parallel with JMeter tests.  There was no quick way to play your selenium test and record a JMeter test.  Problem solved :)

Did you enjoy writing this plugin?

Yes and No.  Yes, had to learn something new.  Once you learn how to integrate with Firefox (or any system) so many other possibilities open up.
No - I had to use the XUL system instead of the update Firefox extension mechanism.

Where is the code?

I made it GPL and opened it up https://github.com/redline13/selenium-jmeter.
Any ideas for extending or contributing would be welcome.

How do I use it?

I wrote the intro and tutorial at RedLine13 blog

  • https://www.redline13.com/blog/2016/01/introducing-selenium-ide-jmeter-plugin
  • https://www.redline13.com/selenium-jmeter-plugin-tutorial

Questions?

If you have any questions feel free to send me a note, but other public mechanisms would be

  • Pull request if you want to add something
  • Ask a question in RedLine13 forum  https://groups.google.com/forum/#!forum/redline13
  • Get a basic RedLine13 subscription and will provide support and get feature requests into the pipeline

Sunday, October 4, 2015

Week in Philly Tech 2015/10/05



Subscribe to Next Week in Philly Tech

 

Monday October 5th

6:00pm Lightning Talks and Workshop with Wine + Code
The Philly Front-end/UX Meetup
Location @ The Hub : 30 South 17th Street, Philadelphia, PA
We're happy to announce a jointly hosted meetup with Wine + Code for October. We'll begin with lightning talks, followed by a workshop session.

  • Jenn Voss: The State of CSS Variables 
  • Miguel Perez: Figuring out task-times without usability testing

Wednesday October 7th

Big Data Developers in Philadelphia
We will be diving deeper into Spark; what it is, what it can do, and how it can provide value to your organization.

Thursday October8th

Tech & GeoPhilly
Location @ Quorum at the Science Center : 3711 Market Street, Philadelphia, PA 19104
Philadelphia's third annual LocationTech event is an evening speaker series featuring technical talks on the convergence of open source and geospatial. This is presented by GeoPhilly and hosted by Azavea.  This event follows the JS.Geo conference in Philadelphia and is conveniently located in the same venue. 

6:00pm Philly CodePen Meetup
Girl Develop It
Location @ Intuitive Company : 3 Rector St., Philadelphia, PA
Gather with your peers, talk shop, and work on Pens with the help of others! Bring your laptop, an idea, and a smile!

6:00pm Exploring the Actor Model for Concurrent Systems using AKKA.NET
Central New Jersey .NET User Group
Location @ Infragistics : 2 Commerce Dr, Cranbury, NJ
In this talk, we'll explore AKKA.NET, a platform with run-time capabilities and behavior inspired by the AKKA framework for JAVA.












Monday, September 28, 2015

Week in Philly Tech 2015/09/28

RedLine13 Newsletter 1.0 


I was a day delayed putting out Next Week In Philly Tech because I was building the first newsletter for RedLine13.   I have read quite a few marketing books including "Traction: A Startup Guide to Getting Customers" by local entrepreneur Gabriel Weinberg and co-author Justin Mares.  While those books have been good, shedding light on the tactics and strategies, they don't always cover time investment and time expectations.

When sharing my thoughts recently with a marketing pro, their response
LOL! Marketing is a lot of work - which is mostly invisible to everyone else!
Then followed by detailed experiences and pitfalls that I appreciated immensely.  As a developer-for-life putting time into marketing is a big change.  I am not sure the following is the most important lesson in marketing, but definitely important for me during this transition.

Software development has an inherent pleasure response - code in, results out, yay.  Marketing is more like planting seeds, watering your plants every day, and one day seeing it just poke out of the dirt.  Exclaim victory but then realize you have so much more to go.   Marketing takes patience, immense investments in time and details while setting personal expectations for one not to get ahead of them self.  All this while no one else really notices.



Subscribe to Next Week in Philly Tech

 

Monday September 28th

Bucks County Women In Tech
Location @ AWeber
We will hear more in depth from 3 WITs about their careers:  
Mali Patel, QA Manager @ LeadiD 
Michele Kopach, Radiologist @ Doylestown Radiology Assoc. 
Amber Heilman, Software Engineer @ AWeber

Tuesday September 29th

Software As A Craft
Location @ PromptWorks : 1211 Chestnut Street Suite 400, Philadelphia, PA 
For September's meeting, we'll focus on the craftsman/apprentice relationship, with A. Jesse Jiryu Davis talking about concrete steps you can take to be an effective mentor.

Linode
Location @ Industrious : 230 S. Broad Street, Philadelphia, PA
Jeremiah is a Technical Solutions Engineer for Puppet Labs.  He is a long time Linux user and most recently has focused on bringing automation solutions into heterogeneous IT environments.

6:30pm Being responsible for engineering quality with Grant Ammons
Bucks County Tech
Location @ MeetMe 100 Union Square Drive, New Hope, PA
In this talk, Grant Ammons focus on how "QA" has turned into a loaded word. Engineering leaders should be wary of creating departments where a "wall" can exist, that the work of quality can be thrown over.

Wednesday September 30th

6:00pm September JavaScript meetup
Philadelphia JavaScript Developers
Location @ Quorum at the Science Center : 3711 Market Street, Philadelphia, PA 19104
How can headless, automated tests be run in continuous integration against technologies that require a real GUI web browser?

6:00pm Android Lightning Talks
Android Alliance
Location @ Comcast Center : 1701 John F. Kennedy Blvd, Philadelphia, PA
This month we are going to have short talks on a variety of topics.

6:30pm NJ Tech Meetup Don Katz CEO Audible
NJ Tech MeetUp
Location @ Audible Inc. : 1 Washington Place, Newark, NJ
Fireside Chat With Don Katz

Thursday October 1st

5:00pm NumPy Creator Travis Oliphant
Philadelphia Python Users Group
Location @ Susquehanna 401 City Avenue, Bala Cynwyd, PA
He will be giving a talk discussing the continuing expansion of the Python community, industry trends, and Continuum Analytics’ role in shaping the future.

6:00pm Code For Philly Workshop #152
Code For Philly
Location @ Devnuts 908 North 3rd Street, Philadelphia, PA
Join us at our weekly workshop to learn about technology, sharpen your skills, and help Philly at the same time! 

6:30pm Ways to build an automation team
Automated Testing Philadelphia
Location @ PromptWorks : 1211 Chestnut Street Suite 400, Philadelphia, PA
One of the hardest parts of bringing automation to an organization can be simply getting started. How do you find the right people? How do you train your current people? Luckily, we have some answers for you! Bob Hefele from CoreDial and John Klose from Swift will each speak about their experiences and challenges in building a QA team focused around automation.

Friday October 2nd

Code For Philly
Location @ Liberty Lands Park - N.3rd St, Poplar St - 
This year N3rd Street officially turns one year old and Code for Philly (as well as many others) are helping celebrate. Check out the website for a comprehensive list of participants: http://n3rd.st

Wednesday October 7th

Big Data Developers in Philadelphia
Location @ IBM Office, Philadelphia, PA

Thursday October8th

Tech & GeoPhilly
Location @ Quorum at the Science Center : 3711 Market Street, Philadelphia, PA 19104
Philadelphia's third annual LocationTech event is an evening speaker series featuring technical talks on the convergence of open source and geospatial. This is presented by GeoPhilly and hosted by Azavea.  This event follows the JS.Geo conference in Philadelphia and is conveniently located in the same venue. 







Monday, September 21, 2015

Next Week in Philly Tech 2015/09/21

Automating for +20 Years

Last week I presented at Tech In Motion Philadephia on Evolution in Automation.  It was a 15-minute presentation slot and my talk was a bit of bridge between Open Source Automation and my specialty in Load Testing.

- Open Source makes automation possible.
- Then proposing a future direction in Automation - From CI to Continuous Automation
- And then integrate my niche of automation - Load Testing.

Truth is Open Source made automation possible for all corners of the engineering organization, but engineers have been automating since day one.

Over 20 years ago I was asked a question - Can you make these? 





And a family business was created.   However,  a few weeks on a band-saw in my garage in the winter was enough to get us thinking - how can we automate this process?  While this was all pre-internet, 3d-printers, and laser-cutters we somehow found 3d-axis machines that were humanly affordable and only programmable in assembly. 

Twenty years later those machines are still in the family though we have moved on from the business.  For me, it was the beginning of learning how to build systematic and automatic systems to make the complicated easy. 



Subscribe to Next Week in Philly Tech

 


Monday September 21st  

6:00pm Major League Sports & Tech
Philly New Tech MeetUp
Location @ The Hub Cira Center : 2929 Arch Street, Philadelphia, PA
We are teaming up with Mobile Monday to host a fantastic sports-themed event that highlights what our local sports teams are doing with mobile technologies to enhance the fan experience and team operations. 

Tuesday September 22nd

8:00am Tech Breakfast  [EventUpon]
Philadelphia TechBreakfast
Location @ LeBow College : 3220 Market Street, Philadelphia, PA

  1. Akili Software, Inc.: Savii Care - Michelle Harper
  2. The World Social Reserve System: - Ronald Williams
  3. TeleStax: Restcomm - Ivelin Ivanov
  4. Rain Everywhere: Rewind - Phil IVes

It may be Pope week in Philly, but we're holding our first Philly PostgreSQL Night on September 22. Unlike our traditional Meetup format, we are extending it to 2 talks for the evening with a sponsored happy hour to follow. 
  • Postgres(QL) in 10
  • Debugging Your PL/pgSQL Code

Wednesday September 23rd

5:30pm Anything C# can do F# can do better [EventUpon]
Philly.NET
Location @ Microsoft : 45 Liberty Blvd, Malvern, PA
Are you a C# developer and have ever wondered what the F# language can do? Or have you wondered if functional programming is really all it’s cracked up to be? 

Thursday September 24th

Avoiding Philadelphia? Head to North Jersey for Google Self-Driving Car Demo
5:00pm Self-Driving Car Demo
Google Developer Group
Location @ Bell Works : 101 Crawfords Corner Rd, Holmdel, NJ
Self-Driving Car Demo (hosted by NJ's own DriveAI guys)

Friday September 25th 

Staying home?  Learn something new for the day.
Take a look at RedLine13 and run a load test today.

Monday September 28th

Bucks County Women In Tech
Location @ AWeber
We will hear more in depth from 3 WITs about their careers:  
Mali Patel, QA Manager @ LeadiD 
Michele Kopach, Radiologist @ Doylestown Radiology Assoc. 
Amber Heilman, Software Engineer @ AWeber

Tuesday September 29th

Software As A Craft
Location @ PromptWorks : 1211 Chestnut Street Suite 400, Philadelphia, PA 
For September's meeting, we'll focus on the craftsman/apprentice relationship, with A. Jesse Jiryu Davis talking about concrete steps you can take to be an effective mentor.

Linode
Location @ Industrious
Jeremiah is a Technical Solutions Engineer for Puppet Labs.  He is a long time Linux user and most recently has focused on bringing automation solutions into heterogeneous IT environments.