Archive Page 4

Simple Environment Handling with Google App Engine

0 Comments Published by January 4th, 2009 in google app engine, technology

Ok, so you’re working with Google App Engine, cool.

It comes time to deploy your application, and of course you’ve created several application copies in your GAE account to have DEV, QA and PROD versions of your app. But how do you manage the app.yaml file which holds the application identifier? (this determines which version is uploaded).

Here’s a simple solution; I have an “upload DEV.bat” file in my root folder that has this simple code:

copy appDEV.yaml app.yaml
appcfg.py update .

When I double click on this file to update my DEV app the script just copies the DEV yaml file to app.yaml and then performs the GAE update. You could replicate this file for every environment you have. Presto.

Deleting inactive WordPressMU blogs

7 Comments Published by December 31st, 2008 in technology, wordpress

Update: we’ve had some problems where some incorrect tables have been deleted by this script. We haven’t had time to investigate, but for now I don’t recommend using this script as is.

Elad posted a small script to delete inactive WordressMU blogs. This can be useful if you run a blog network and want to automate the removal of stale content and regain some database space. I recommend backing up your DB regularly if you plan on running this script in production as it deletes data from the DB.

AJ and I made some modification and improvements to Elad’s version. Namely:

  • fixed bugs in the original (swapped blogs_mail and blogs_delete values)
  • added option to exclude certain blogs from the process (so that you don’t accidentally delete your primary blog for example)
  • changed mail to use SMTP in case you don’t run a mail server on your WordPressMU box
  • changed delete procedure to loop though all user tables (no longer hard-coded to only certain tables)

To use this code place the .php file somewhere inside your wordpressMU install and setup a cron job to run it weekly (for example, something like “php -q /path/wpmu-cleanup.php”). Note that this code does not delete the user from the wp_users table, nor does it delete their folder in /wp-content/blogs.dir.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211

< ?php
/**********************************************************
WPMU Cleanup
Original Version by Elad Salomons: http://www.ringofblogs.com/2008/06/10/wpmu-cleanup-script-to-delete-unused-blogs/
Modified by Boris Masis: http://www.borism.net/2008/12/31/deleting-inactive-wordpressmu-blogs
 
This script checks the WPMU database for blogs that have
not been updated for X days, then sends a notice to the
blogowner that the blog will be deleted after Y days.
 
It also automatically deletes blogs that have not been
updated for Y days, then sends an email to the blogowner.
 
On both occasions, the script will send a report with
details on warned and deleted blogs to the administrator.
 
This script has *NOT* been programmed as a plugin since
it is intended to be executed by a cronjob. As it is now
it should be run once every 7 days, but you can configure
it to do otherwise.
 
This script is *NOT* a final version and should be used
with caution and only if you know what you are doing. I
recommend testing it on a testsite rather than a livesite.
 
**********************************************************/
 
/* SETTINGS */
require_once "Mail.php";
require_once('wp-load.php');
$mysql_server				= '';
$mysql_username				= '';
$mysql_password				= '';
$mysql_database				= 'wordpress';
 
$options['email_host']		= '';
$options['email_from']		= ''; // Email-address used for sending notices.
$options['email_subject']	= 'Your Website';
$options['mail_report']		= ''; // Reports will be sent to this address.
$options['timetomail']		= 256; // Number of days blog can be inactive before receiving deletion warning.
$options['timetodeletion']	= 270; // Number of days blog can be inactive before deletion.
$options['dateformat']		= 'm-d-Y'; // Dateformat used in emails.
 
$options['noticemessage']	= 'Dear user,' . "\n\n" . 'You have not updated your website in %timetomail% days. This message is to remind you that your website will be deleted if it is not updated in %timetodeletion% days.' . "\n\n" . 'If you would like to keep your website, please update it before %date%.' . "\n\n" . 'Best regards.' . "\n\n";
$options['deletemessage']	= 'Dear user,' . "\n\n" . 'Since your website has not been updated for %timetodeletion% days, it has been deleted.' . "\n\n" . 'Please contact us if you have any questions.' . "\n\n" . 'Best regards.' . "\n\n";
 
$options['blogstoignore']	= array(); //blogs that should not be part of this logic for example =array(1,2);
 
/********************/
/*** STOP EDITING ***/
/********************/
 
$options['unix_timetomail']		= $options['timetomail'] * 24 * 60 * 60;
$options['unix_timetodeletion']	= $options['timetodeletion'] * 24 * 60 * 60;
$options['date']				= date($options['dateformat'], (($options['timetodeletion'] - $options['timetomail']) * 24 * 60 * 60) + time());
 
function mailer ($to, $from, $subject, $body){
	global $options;
	$headers = array ('From' => $from,
	  'To' => $to,
	  'Subject' => $subject);
	$smtp = Mail::factory('smtp',
		array ('host' => $options['email_host'],
		'auth' => false));
	$mail = $smtp->send($to, $headers, $body);
}
 
// Connect to database
$db =	mysql_connect($mysql_server,
					   $mysql_username,
					   $mysql_password);
 
		mysql_select_db($mysql_database, $db);
 
// Get blogs
$sql = 'SELECT blog_id,last_updated FROM wp_blogs';
$sql = mysql_query($sql);
 
		while( $row = mysql_fetch_assoc($sql) ) :
 
			if (!in_array($row['blog_id'], $options['blogstoignore']))
				$blogs_raw[$row['blog_id']] = strtotime($row['last_updated']);
 
		endwhile;
 
		// Get blogs for deletion notice
		foreach( $blogs_raw as $key => $value ) :
 
			$time = time() - $value;
 
			if( $time >= $options['unix_timetomail'] && $time < $options['unix_timetodeletion'] ) :
 
				$blogs_mail[] = array('blogid'  => $key,
									  'email'   => '',
									  'siteurl' => '');
 
			endif;
 
		endforeach;
 
				// Get userdata for warning notice
				for($i = 0; $i < sizeof($blogs_mail); $i++) :
 
					$str = 'admin_email';
					$sql = mysql_query("SELECT option_value FROM wp_" . $blogs_mail[$i]['blogid'] . "_options WHERE option_name = '$str'");
 
						while ( $row = mysql_fetch_assoc($sql) ) :
 
							$blogs_mail[$i]['email'] = $row['option_value'];
 
						endwhile;
 
					$str = 'siteurl';
					$sql = mysql_query("SELECT option_value FROM wp_" . $blogs_mail[$i]['blogid'] . "_options WHERE option_name = '$str'");
 
						while ( $row = mysql_fetch_assoc($sql) ) :
 
							$blogs_mail[$i]['siteurl'] = $row['option_value'];
 
						endwhile;
 
				endfor;
 
						$cronmsg = 'The following blogs have been sent deletion warnings:' . "\n\n";
						// Create and send warning notice to users
						for($i = 0; $i < sizeof($blogs_mail); $i++) :
 
							$message = str_replace('%timetomail%', $options['timetomail'], $options['noticemessage']);
							$message = str_replace('%timetodeletion%', $options['timetodeletion'], $message);
							$message = str_replace('%date%', $options['date'], $message);
							//$message = str_replace('%loginurl%', $blogs_mail[$i]['siteurl'] . 'wp-login.php', $message);
 
							mailer($blogs_mail[$i]['email'], $options['email_from'], $options['email_subject'], $message);
 
							$cronmsg .= $blogs_mail[$i]['siteurl'] . "\n";
 
						endfor;
						if (sizeof($blogs_mail) > 0) mailer($options['mail_report'], $options['mail_report'], 'Blogs Sent Warnings', $cronmsg);
 
 
 
		// Get blogs for deletion
		foreach( $blogs_raw as $key => $value ) :
 
			$time = time() - $value;
 
			if( $time >= $options['unix_timetodeletion'] ) :
 
				$blogs_delete[] = array('blogid'  => $key,
										'email'   => '',
										'siteurl' => '');
 
			endif;
 
		endforeach;
 
				// Get userdata for deletion notice
				for($i = 0; $i < sizeof($blogs_delete); $i++) :
 
					$str = 'admin_email';
					$sql = mysql_query("SELECT option_value FROM wp_" . $blogs_delete[$i]['blogid'] . "_options WHERE option_name = '$str'");
 
						while ( $row = mysql_fetch_assoc($sql) ) :
 
							$blogs_delete[$i]['email'] = $row['option_value'];
 
						endwhile;
 
					$str = 'siteurl';
					$sql = mysql_query("SELECT option_value FROM wp_" . $blogs_delete[$i]['blogid'] . "_options WHERE option_name = '$str'");
 
						while ( $row = mysql_fetch_assoc($sql) ) :
 
							$blogs_delete[$i]['siteurl'] = $row['option_value'];
 
						endwhile;
 
				endfor;
 
						// Delete tables
						$cronmsg = 'The following blogs have been deleted:' . "\n\n";
						for($i = 0; $i < sizeof($blogs_delete); $i++) :
 
							//loop though and delete all tables that begin with the blog id
							$prefix = 'wp_' . $blogs_delete[$i]['blogid'] . '_%';
 
							$querystr ="SHOW TABLES LIKE '$prefix'";
							$tables = $wpdb->get_results($querystr, ARRAY_N);
 
							$query = "";
							if ($tables){
								foreach ($tables as $table){
									$query = 'DROP TABLE IF EXISTS '. $table[0];
									$wpdb->query($query);
								}
							}
 
							$sql_blogid	= $blogs_delete[$i]['blogid'];
							mysql_query("DELETE FROM wp_blogs WHERE blog_id = '$sql_blogid'") or die('Error: ' . mysql_error());
 
							// Send mail
							$message = str_replace('%timetodeletion%', $options['timetodeletion'], $options['deletemessage']);
 
							mailer($blogs_delete[$i]['email'], $options['email_from'], $options['email_subject'], $message);
 
							$cronmsg .= $blogs_delete[$i]['siteurl'] . "\n";
 
						endfor;
						if (sizeof($blogs_delete) > 0) mailer($options['mail_report'], $options['mail_report'], 'Deleted Blogs', $cronmsg);
 
?>

Integrating Google Analytics and Wufoo

42 Comments Published by December 24th, 2008 in google analytics, technology, wufoo

It’s fairly simple to integrate Google Analytics so that your campaign variables get saved behind the scenes when someone submits your wufoo form.

The code is based on the original post at analytics talk, with some small modifications for wufoo.

All we need to do is include a small javascript file that will parse our google analytics cookie and pass it to wufoo using url modification.

Here’s a 4 minute screencast of what we’ll be achieving:

Note that the code relies on the old version of the google analytics code (urchin.js update: you can find updated javascript for the new version of the tracking code at Analytics Talk, however I have not yet integrated it with wufoo) and I’m using jquery for $(document).ready functionality, but you could use your library of choice if you don’t use jQuery.

The complete javascript is

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

$(document).ready(function(){
    //
    // Get the __utmz cookie value. This is the cookies that
    // stores all campaign information.
    //
    var z = _uGC(document.cookie, '__utmz=', ';');
    //
    // The cookie has a number of name-value pairs.
    // Each identifies an aspect of the campaign.
    //
    // utmcsr  = campaign source
    // utmcmd  = campaign medium
    // utmctr  = campaign term (keyword)
    // utmcct  = campaign content (used for A/B testing)
    // utmccn  = campaign name
    // utmgclid = unique identifier used when AdWords auto tagging is enabled
    //
    // This is very basic code. It separates the campaign-tracking cookie
    // and populates a variable with each piece of campaign info.
    //
    var source  = _uGC(z, 'utmcsr=', '|');
    var medium  = _uGC(z, 'utmcmd=', '|');
    var term    = _uGC(z, 'utmctr=', '|');
    var content = _uGC(z, 'utmcct=', '|');
    var campaign = _uGC(z, 'utmccn=', '|');
    var gclid   = _uGC(z, 'utmgclid=', '|');
    //
    // The gclid is ONLY present when auto tagging has been enabled.
    // All other variables, except the term variable, will be '(not set)'.
    // Because the gclid is only present for Google AdWords we can
    // populate some other variables that would normally
    // be left blank.
    //
    if (gclid !="-") {
          source = 'google';
          medium = 'cpc';
    }
    // Data from the custom segmentation cookie can also be passed
    // back to wufoo via url modification
    var csegment = _uGC(document.cookie, '__utmv=', ';');
    if (csegment != '-') {
          var csegmentex = /[1-9]*?\.(.*)/;
          csegment    = csegment.match(csegmentex);
          csegment    = csegment[1];
    } else {
          csegment = '';
    }
    //modify the wufoo iframe location to pass the analytics values
    frames[0].location.href = 'https://youraccount.wufoo.com/embed/123/def/field30=' + source + '&field36=' + medium + '&field35=' + term + '&field34=' + content + '&field33=' + campaign + '&field32=' + gclid;
 
});

Fixing a Large CMSPropertyData Table in Umbraco

0 Comments Published by December 16th, 2008 in umbraco

Current versions of Umbraco appear to have a bug that may cause corruption of publishing times and cause documents to re-publish themselves many times over. This can lead to an extremely large CMSPropertyData table and the need for frequent application pool recycles. See http://forum.umbraco.org/yaf_postst4699p2_Extremely-large-Umbraco-database-55GB-table-cmsPropertyData-contains-over-135-million-records.aspx and http://forum.umbraco.org/yaf_postst6564_305-Database-Huge–cmsPropertyData–392MB.aspx for a discussion of the issue.

We’ve done several things to control this issue, though a true fix has yet to be developed:

  1. We applied the suggested fix of changing the umbraco.presentation.cache.CacheRefresher() constructor (this seemed to have no effect): 
    public CacheRefresher() {
   if (System.Web.HttpContext.Current != null)
    {
        this.Url = "http://" + System.Web.HttpContext.Current.Request.ServerVariables["SERVER_NAME"] + "/umbraco/webservices/cacheRefresher.asmx";
    }
}
  2. We removed the future publishing functionality from the UI (umbraco/presentation/umbraco/editContent.aspx.cs in umbraco 3.03): 
    //dpRelease.ID = "releaseDate";
//dpRelease.DateTime = _document.ReleaseDate;
//dpRelease.ShowTime = true;
//publishProps.addProperty(ui.Text("content", "releaseDate", base.getUser()), dpRelease);
//dpExpire.ID = "expireDate";
//dpExpire.DateTime = _document.ExpireDate;
//dpExpire.ShowTime = true;
//publishProps.addProperty(ui.Text("content", "expireDate", base.getUser()), dpExpire);
  3. We commended out the publishing timer which seems to have stopped the growth without adverse effects (thought I imagine that scheduled publishing would no longer work) (/umbraco/presentation/requestModule.cs in umbraco 3.03) 
    //publishingTimer =
//    new Timer(new TimerCallback(publishingService.CheckPublishing), HttpApp.Context, 600000, 60000);

The author of ClientTools promises to release a new version which should be able to cleanup the large number of item revisions in the database.

Stored procedures useful for troubleshooting this issue include:

  • find items with the most revisions: 
    SELECT contentNodeId, COUNT(contentNodeId) FROM cmsPropertyData GROUP BY contentNodeId ORDER BY COUNT(contentNodeid) DESC
  • manually delete an item (if you can’t find the item in the UI using http://site/umbraco/editContent.aspx?id=IDHERE): 
    DELETE FROM cmsPropertyData WHERE contentNodeId = '4023'
  • find lost documents: 
    SELECT * FROM umbracoNode
WHERE nodeObjectType = 'c66ba18e-eaf3-4cff-8a22-41b16d66a972' AND ID NOT IN (SELECT Nodeid FROM cmsDocument)
  • find items scheduled to publish: 
    SELECT DISTINCT nodeId, level, sortOrder FROM cmsDocument INNER JOIN umbracoNode ON umbracoNode.id = cmsDocument.nodeId WHERE newest = 1 AND NOT releaseDate IS NULL AND releaseDate < = getdate() ORDER BY level, sortOrder

Changing Domains/Environments in WordPress MU

4 Comments Published by December 2nd, 2008 in technology, wordpress

I’m working on a WordPress MU blog network and ran accross the need to change the domain that the blogs are running under. This should actually be a fairly common need, for example, if you copy your PROD environment to QA, you’ll probably want to change the blog url from http://blog.com to http://qa.blog.com.

Changing the domain name starts out pretty straightforward:

  • edit the “domain” in the wp_site table.
  • edit all “domain” entries in the wp_blogs table: UPDATE wp_blogs SET domain = REPLACE(domain,”blog.com”,”qa.blog.com”)

But then, you’ll notice that there’s a lot of hard-coded urls in the wp_#_options table, which exists for every blog in your network.

So, here’s a little PHP file that you can put in the root of your WordPress MU install (or elsewhere). This will loop though these tables and do the replaces you need:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

< ?php
require_once('wp-load.php');
 
$querystr ="SHOW TABLES LIKE 'wp_%_options'";
$tables = $wpdb->get_results($querystr, ARRAY_N);
echo count($tables);
$query = "";
if ($tables){
	foreach ($tables as $table){
		$query = 'UPDATE '.$table[0].' SET option_value = REPLACE(option_value,"blog.com","qa.blog.com")';
		$wpdb->query($query);
	}
}
?>

The file will output a count of the number of tables that were affected. Ta da

URL Rewriting with IIS and Umbraco

1 Comment Published by November 26th, 2008 in technology, umbraco

At Cultural Care Au Pair we use Google Analytics to track the sucess of our campaigns both online and offline. Google provides a nice url builder tool that generates links tagged with our various campaign attributes. Here’s the problem though, the links end up looking like:

http://www.culturalcare.com/default.aspx?utm_source=testSource&utm_campaign=testCampaign

That’s quite an ugly url for something like a postcard campaign. Luckily umbraco includes a full url rewriter that can help us easily transform that into something like www.culturalcare.com/postcard

1. Configure IIS for wildcard application mapping. This allows ASP.NET to process extension-less urls (such as /postcard/):

  • right click on the umbraco website and select properties
  • go to the “home directory” tab and click “configuration”
  • in the “wildcard application maps” insert the ASP.NET dll (something like C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\aspnet_isapi.dll ) and un-check “verify that file exists”

2. Edit /config/UrlRewriting.config in your umbraco folder

  • Change the top tag to ensure that you don’t have any conflicts with other extension-less urls (like /umbraco) 
    <urlrewritingnet rewriteOnlyVirtualUrls="true"
contextItemsPrefix="QueryString"
defaultPage = "default.aspx"
defaultProvider="RegEx"
xmlns="http://www.urlrewriting.net/schemas/config/2006/07" ><rewrites>
...
</rewrites>
</urlrewritingnet>
  • add your rewrite rules following the example in the file or the documentation at urlrewriting.net. For example, if I wanted to redirect /postcard/ to /default.aspx?utm_source=testSource&utm_campaign=testCampaign I would add the following rule: 
    <add name="postcard"
virtualUrl="^~/postcard/?$"
redirect="Application"
rewriteUrlParameter="ExcludeFromClientQueryString"
destinationUrl="~/default.aspx?utm_source=testSource&amp;utm_campaign=testCampaign"
ignoreCase="true"
/>
  • Note that the & needs to be written as &amp; in the destination url.
  • If you’d like to redirect to a different domain, virtual url and redirect need to be specified accordingly: 
    <add name="offsite"
virtualUrl="^http\://(.*)my.domain/redirect/?$"
redirect="Domain"
rewriteUrlParameter="ExcludeFromClientQueryString"
destinationUrl="http://offsitedomain.com"
ignoreCase="true"
/>

Enjoy!

This Aint a Blog

0 Comments Published by August 5th, 2007 in site news

Who am I kidding with posts like “Film Recommendation”? I don’t actually like to write updates very much, but here are the things that actually do change around here:

  • photos.borism.net is my friend.
  • refresh a couple of times to see different header images. There are now 202 in rotation.
  • my bookmarks and calendar in the right sidebar update themselves, hooray!

It's Been a Long Weekend

0 Comments Published by December 26th, 2006 in site news

I added some more photos into the header rotation. Mostly they are from Sam and my trip to Spain and Morocco; hit refresh a couple times. 🙂

Film Recommendation

6 Comments Published by June 27th, 2006 in culture

Film recommendation for everyone: go see word play. It’s a genuinely enjoyable film. After you get back, you can follow up with a nice read.

New Alarm Time = 5:30 AM

2 Comments Published by May 29th, 2006 in productivity

The internet is so full of good self improvement advice, its scary. In any case, my new alarm time is 5:30 AM thanks to How to Become an Early Riser. Today I got up at that un-godly hour out of my own volition. That means that before work I had time to:

  • sort laundry
  • mail a package
  • take some for-sale photos for craigslist
  • prepare my breakfast and lunch
  • and walk around the apartment (quietly)

I think at this rate I’ll start running out of things to do. OK, OK, this was just day one. Wish me luck.

« Previous Entries
Next Entries »

Search


About

Hi, I'm Boris. This is my website.


Archives

Categories