Developer's Closet A place where I can put my PHP, SQL, Perl, JavaScript, and VBScript code.

29Sep/170
Use PowerShell to Retrieve a List of Servers from VMM

Useful script to go after servers in VMM and output the list to a csv file.

Import-Module virtualmachinemanager;
$MasterList = @();
$serverList = Get-VM -VMMServer seacorpmgmt05 | Where-Object { (($_.Name -like '*DEMO*' -or $_.Name -like '*APP*' -or $_.Name -like '*WEB*' -or $_.Name -like '*CORP*' -or $_.Name -like '*UTIL*' -or $_.Name -like '*MGMT*') -and $_.Name -notlike '*dev*') };
foreach ($server in $serverList) {
   $serverNetworkAdapter = Get-SCVirtualNetworkAdapter -VM $server;
   $MyObject = New-Object PSObject -Property @{
      ServerName = $server.Name;
      ServerEnvironment = $server.Name.Substring(3,4);
      HostName = $server.HostName;
      VMStatusString = [string]$server.StatusString;
      VMState = [string]$server.VirtualMachineState;
      ServerIPAddress = $serverNetworkAdapter.IPv4Addresses;
      CPUCount = $server.CPUCount;
      Memory = $server.Memory;
      OperatingSystem = $server.OperatingSystem;
      IsHighlyAvailable = $server.IsHighlyAvailable;
      Description = $server.Description.Replace([Environment]::NewLine, '');
      CreationTime = $server.CreationTime;
   };
   $MasterList += $MyObject;
};
$MasterList | Select-Object @{label='Server Name';expression={$_.ServerName.ToUpper()}}, @{label='Environment';expression={$_.ServerEnvironment.ToUpper()}}, @{label='VM Status';expression={if ($_.VMStatusString -eq $_.VMState) { $_.VMStatusString } else { $_.VMState + ' (' + $_.VMStatusString + ')'}}}, @{label='Host Name';expression={$_.HostName.Replace('.ds-iq.corp','').ToUpper()}},@{Name=’ServerIPAddress’;Expression={if ([string]::join(";", ($_.ServerIPAddress))) {[string]::join(";", ($_.ServerIPAddress))} else { " " }}},@{label='Cores';expression={$_.CPUCount}},@{label='Memory';expression={$_.Memory}},@{label='OS';expression={$_.OperatingSystem}},@{label='Highly Available';expression={$_.IsHighlyAvailable}},@{label='Comments';expression={if ($_.Description) {$_.Description + ', Created ' + $_.CreationTime} else { 'Created ' + $_.CreationTime }}} | EXPORT-CSV c:\temp\app-server-2.csv -notype;
26Sep/170
Search for string and append string to file

A very useful string search and append that I find myself rewriting over and over. So I've decided to write a post and capture the script:

if grep -q sensu-client "/etc/rc.local"; then
echo "exists";
else
echo "does not exist"
sudo sed -i '/exit 0$/i sudo service sensu-client restart' /etc/rc.local;
fi

 

Filed under: Bash, Linux, Ubuntu No Comments
19Apr/160
Manage Traces in SQL Server

Quick notes on managing traces in SQL Server:

View all running traces:

SELECT * FROM sys.traces

Disable and delete the trace with the @status option = 2:

EXEC sp_trace_setstatus @traceid = 2 , @status = 0
EXEC sp_trace_setstatus @traceid = 2 , @status = 2

Use option 0 = disable, 1 = start, 2 = disable and delete.

Filed under: SQL No Comments
21May/150
Delete a Kafka Topic

Typical delete command:

kafka-topics --zookeeper zk_host:port/chroot --delete --topic my_topic_name

The topic will be "marked for deletion" and should be removed soon. What happens is an /admin/delete_topics/<topic> node is created in zookeeper and triggers a deletion. When the broker sees this update, the topic will no longer accept a new produce/consume request and eventually the topic will be deleted. If the delete command doesn't work right away, try restarting the Kafka service.

To delete a Kafka topic after the Broker has lost connection to the topic:

Manually delete the topic:

If the delete commands fails (marked for deletion forever):

1. Stop the Kafka Broker.

2. Delete the topic from disk.

For example:

sudo mv /space1/kafka/FourSquareSearchApiResults-0 /space1/kafka/FourSquareSearchApiResults-0.bak

3. Delete the topic znode from ZooKeeper:

 

Here are the ZooKeeper commands to use to delete the topic from ZK:

1. Log onto zkcli:

hbase zkcli

2. Search for topics:

ls /brokers/topics

rmr /brokers/topics/my_topic_name

ls /brokers/topics

3. Start Kafka.

4. You can then recreate the Kafka topic.

21Apr/150
Bash

Output in red:

echo "$(tput setaf 1)Red Text$(tput sgr0)"

Filed under: Linux No Comments
26Feb/150
PowerShell script to convert string to encrypted password and back again

I've found it useful to store an encrypted password in a file on disk. However, this is not generally a good idea but at least the password is encrypted to the user/machine. Here is a rough outline of the process:

-- convert to encrypted password from clear text
$password = ConvertTo-SecureString 'SomePassword' -AsPlainText –Force
-- write to file as user/machine encrypted string which cannot be unencrypted by any other user on any other machine
$password|convertFrom-SecureString|set-content "c:\temp\cred.txt"
$password

 

To make the file really useable, preface the password with the username and query for the username:

-- query for user name from file

$username = "UserName";
-- convert password to a secure string that can be sent through to Windows for authentication
$password = Get-Content "c:\temp\cred.txt" | Select-String $Username | foreach{$_ -replace $Username, ""} | foreach{$_ -replace " ",""} | ConvertTo-SecureString;

 

Finally, you can go a step further and crack open the password:

-- convert from encrypted password to clear text
$Ptr = [System.Runtime.InteropServices.Marshal]::SecureStringToCoTaskMemUnicode($password);
$password = [System.Runtime.InteropServices.Marshal]::PtrToStringUni($Ptr);[System.Runtime.InteropServices.Marshal]::ZeroFreeCoTaskMemUnicode($Ptr);
$password
Filed under: PowerShell No Comments
9Feb/150
Find and Delete Running SQL Trace

Yesterday we had a long running trace take up too much space on disk. To find the running trace, look for the location of the file to identify your trace, and get the id:

SELECT * FROM sys.traces

Next, disable and delete the trace with the @status option = 2:

EXEC sp_trace_setstatus @traceid = # , @status = 2

Use option 0 = disable, 1 = start, 2 = disable and delete.

Have fun with traces, but make sure to stop them before you go home for the weekend!

Filed under: SQL No Comments
9Sep/140
Recover Lost Files on Linux

TestDisk includes PhotoRec which is very good at recovering files:

sudo apt-get install testdisk

To run PhotoRec:

sudo photorec

PhotoRec: http://www.cgsecurity.org/wiki/PhotoRec

 

Filed under: Linux, Ubuntu No Comments
6Aug/140
Copy data from one Hadoop cluster to another Hadoop cluster (running different versions of Hadoop)

I had to copy data from one Hadoop cluster to another recently. However, the two clusters ran different versions of Hadoop, which made using distcp a little tricky.

Some notes of distcp: By default, distcp will skip files that already exist in the destination, but they can be overwritten by supplying the -overwrite option. You can also update only files that have changed using the -update option. distcp is implemented as a MapReduce job where the work of copying is done by maps that run in parallel across the cluster. There are no reducers. Each file is copied by a single map, and distcp tries to give each map approximately the same amount of data, by bucketing files into roughly equal allocations.

The following command will copy the folder contents from one Hadoop cluster to a folder on another Hadoop cluster. Using hftp is necessary because the clusters run a different version of Hadoop. The command must be run on the destination cluster. Be sure your user has access to write to the destination folder.

hadoop distcp -pb hftp://namenode:50070/tmp/* hdfs://namenode/tmp/

Note: The -pb option will preserve the block size.

Double Note: For copying between two different versions of Hadoop we must use the HftpFileSystem, which is a read-only files system. So the distcp must be run on the destination cluster.

The following command will copy data from Hadoop clusters that are the same version.

hadoop distcp -pb hdfs://namenode/tmp/* hdfs://namenode/tmp/

Filed under: HDFS, Ubuntu No Comments
4Aug/140
Useful Scripts

PowerShell:

$ou = [adsi]"LDAP://OU=Marketing,OU=Departments,DC=Company,DC=Domain";
$user = $ou.psbase.get_children().find('CN=UserName');
$user.psbase.invokeSet("allowLogon",0);
$user.setinfo();

Bash:

#!/bin/bash
fname="/path/file"
tname="/new/file.tmp"
i="0"
DATE=$(date +%Y%m%d%H%M%S)
sudo cp "$fname" "$fname".$DATE

while IFS='' read -r line
do
if [ "$line" == " line item:" ] || [ $i -gt 0 -a $i -lt 4 ]
then
echo -e "#$line" | sudo tee -a $tname
i=$(($i + 1))
else
printf "%s\n" "$line" | sudo tee -a "$tname"
fi
done <"$fname"

sudo mv $tname $fname

Batch:

$SourcePath = 'C:\Path\';
$DestServer = 'ServerName';
$DestPath = '/path/';
$FileName = 'FileName';
$Output = @()
$cmd = @(
"y",
"lcd $SourcePath",
"cd $DestPath",
"mput $FileName",
"quit"
)

$Output = $cmd | & "C:\Program Files (x86)\Putty\psftp.exe" –v $DestServer 2>&1;
$Err = [String]($Output -like "*=>*");
If (($LastExitCode -ne 0) || (($Err.Contains("=>")) -eq $false)) {
throw "File Failed to Transfer! `n $($Output)";
}

Linux:

sudo mkdir /space;
echo "/dev/space /space ext4 defaults 0 0" | sudo tee -a /etc/fstab;
sudo mount /dev/space /space;
sudo df -h;
ls /dev/;

PowerShell:

Add-PSSnapin Quest.ActiveRoles.ADManagement;
connect-QADService -service domain;
set-QADuser UserName -TSRemoteControl 0;
$objCurrentPSProcess = [System.Diagnostics.Process]::GetCurrentProcess();
Stop-Process -Id $objCurrentPSProcess.ID;